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ABSTRACT 


TRANSFORMING  THE  CONDUCT  OF  NAVAL  STRIKE  FIGHTER  OPERATIONAL 
TEST  AND  EVALUATION,  by  Lieutenant  Commander  Erie  L.  Taylor,  United  States 
Navy,  89  pages. 

Aequisition  reform  has  been  a  eontinual  proeess  for  the  past  230  years.  Today’s  reforms 
eontinue  that  trend.  In  the  era  of  redueing  budgets  and  inereasing  aequisition  cost,  a 
solution  must  be  found  to  reform  the  current  structure.  The  Department  of  Defense  has 
initiated  such  reform  goals  in  its  transformation  initiatives.  However,  these  solutions  are 
still  running  into  the  problems  of  previous  reform  efforts. 

The  acquisition  system  is  enormous  and  any  attempt  to  fix  the  entire  system  is  beyond  the 
scope  of  this  paper.  This  paper  will  focus  more  on  a  micro  level,  in  particular  the 
relationship  between  the  Developmental  Test  and  Evaluation  (DT&E)  and  Operational 
Test  and  Evaluation  (OT&E)  squadrons  located  at  Naval  Air  Weapons  Station  (NAWS) 
China  Lake,  California. 

The  relationship  between  DT&E  and  OT&E  is  strained,  causing  problems  in  cost, 
schedule,  and  communication.  Additionally,  personnel  staffing  and  an  antiquated  test 
structure  creates  problems  which  reduce  efficiency.  The  differing  goals  and  priorities  of 
the  institutions  involved  accompanied  with  the  separate  command  structures  creates  this 
friction.  These  differences  will  be  addressed  in  detail.  The  solution  presented  and 
analyzed  in  this  paper  proposes  combining  these  two  squadrons  into  one  squadron. 
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CHAPTER  1 


INTRODUCTION 

Introduction 

Acquisition  reform  has  been  a  eontinual  proeess  ever  sinee  the  first  item  was 
purchased  by  a  government  agency.  With  the  birth  of  the  United  States  in  the  latter  half 
of  the  18*  eentury  eame  the  need  for  a  new  government  to  procure  items  and  goods. 
Before  long,  the  new  government  enaeted  laws  to  define  how  and  what  goods  to  aequire. 
With  these  laws  came  the  birth  of  the  U.S.  aequisition  system.  Since  then  laws  and 
regulations  have  been  added  on  a  continual  basis  crafting  the  modem  day  acquisition 
system.  Yet  as  time  passed,  the  system  has  morphed  itself  into  a  monstrous  bureaucratic 
process  while  eries  for  reform  echo  constantly  through  the  streets  and  buildings  of  the 
nation’s  eapital.'  It  is  likely  that  these  cries  of  reform  ean  also  trace  their  roots  back  to 
the  first  laws  enacted  to  regulate  government  aequisition. 

Acquisition  reform  has  taken  on  the  guise  of  a  never  ending  series  of  legislative 
efforts  and  changes  in  the  way  of  doing  business.  Reform  itself  has  almost  beeome  a 
business.  The  modern  era  of  aequisition  reform  began  with  the  proeesses  that  resulted 
from  the  Vietnam  War  acquisition  programs.  Since  then  a  continual  scmtiny  has  been 
plaeed  on  how  the  Department  of  Defense  (DoD)  aequires  new  systems.  While  various 
administrations  and  policies  have  eome  and  gone  since  Vietnam,  the  one  constant  has 
been  aequisition  reform.  Figure  1  outlines  a  brief  synopsis  of  the  reform  initiatives  taken 
over  the  past  50  years.  It  by  no  means  encompasses  the  entire  reform  effort  but  provides  a 
general  idea  on  the  size  and  eomplexity  assoeiated  with  aequisition  reform. 
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Year 

Regulation  or  Initiative  Published 

1969 

Packard  Initiatives 

1971 

niuo  Ribbon  Dofnnso  Panol  (rit7hugli  Commission) 

197? 

DoDD  5000.1  (Major  System  Acquisitions);  Commission  on  Covertiment 
Procurement 

1973 

DoDD  5000.4  (CAIC);  DoDD  5000.3  (T&r) 

1975 

DoDI  5000.?  (Major  System  Acquisitions);  DoDD  5000.?6  (DTC) 

1976 

OMB  Circular  A  109 

1976 

Defense  Science  Board  Acquisition  Cycle  Task  T orce 

1979 

Defense  Resource  Management  Study 

1961 

Carlucci  Initiatives;  Defense  Acquisition  Improvement  Program 

198? 

Nunn-McCurdy  (thresholds) 

1983 

Grace  Commission 

1985 

DoD  5000.43  (slrearnlinlng) 

1986 

Packard  Commission 

1987 

DoDD  5134.1  (USD(A);  DoDD  5000.49  (DAB) 

1989 

Defense  Management  Review 

1991 

Revised  DoDI  5(X10.2  (Major  System  Acquisitions) 

1994 

Federal  Acquisition  Streamlininq  Act  (FASA) 

1995 

Federal  Acquisition  Improvement  Act  (FASA  II) 

Figure  1.  Defense  Acquisition  Reform  Initiatives  to  1995 


Source:  Christensen,  David  S.  Ph.D.,  CAPT  David  A.  Searle  USAF,  and  Dr.  Caisse  Vickery. 
"The  Impact  of  the  Packard  Commission's  Recommendations  on  Reducing  Cost  Overruns  in 
Major  Defense  Acquisition  Programs."  Acquisition  Review  Quarterly.  (Summer  1999):  253. 


Congress  regulates  everything  in  the  aequisition  system  with  the  laws  it  enacts 
and  the  budgets  it  passes.  The  acquisition  system  reflects  the  government  and  political 
system  in  which  it  exists.  Originally  instituted  as  a  system  of  checks  and  balances,  the 
various  branches  of  government  hamper  efficiency  and  reform.  However,  the  power 
Congress  wields  is  a  necessary  one.  While  the  regulations  and  bureaucracy  placed  on  the 
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acquisition  system  are  burdensome,  they  do  provide  some  attempt  to  improve  the  system. 
Getting  new  changes  enaeted  into  law  through  Congress  have  proven  to  be  difficult,  yet 
some  changes  can  be  made  within  the  confines  of  the  current  laws  that  do  not  require 
Congressional  action. 

The  defense  acquisition  system  is  a  behemoth.  People  spend  lifetimes  learning  the 
inner  workings  of  the  system  and  never  seem  to  understand  the  systems’  inner  workings. 
The  number  of  reforms  alone  suggest  an  enormity  all  its  own.  Some  have  succeeded  in  a 
small  way  while  others  have  failed.  The  intent  of  this  paper  is  to  suggest  another  possible 
reform  of  a  smaller  element  in  the  aequisition  eommunity.  Speeifically  this  paper  will 
focus  on  the  relationship  between  the  Developmental  Test  and  Evaluation  (DT&E)  and 
Operational  Test  and  Evaluation  (OT&E)  communities.  United  States  Code  Title  10  2399 
(2007)  draws  a  elear  distinetion  between  the  two  types  of  test  organizations.  DT&E  tests 
systems  and  subsystems  against  engineering  and  contraet  specifications,  whereas  OT&E 
tests  the  end  produet  of  the  programs  against  mission  requirements.  Historieally  these 
two  entities  have  been  separated  throughout  the  aequisition  process.  While  there  have 
been  attempts  to  draw  these  two  closer  together  in  recent  years,  the  relationship  is  still 
distant,  causing  friction  and  duplication  of  effort  resulting  in  inereased  cost  and  time  to 
deployment  for  new  weapons  systems  and  capabilities.  While  this  separation  is  mainly 
driven  by  law,  some  changes  can  be  made  within  the  current  structure  to  improve 
efficieney  and  reduce  cost. 

The  reduction  of  program  cost  is  an  absolute  necessity  in  modern  aequisition.  The 
length  of  time  programs  spend  in  the  aequisition  pipeline  directly  affects  the  eost  of  the 
overall  system.  The  average  length  of  time  it  has  taken  for  an  aireraft  program  to  move 
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through  the  acquisition  process  to  deployment  can  take  a  decade  or  more.  Additionally, 
by  the  time  most  of  the  systems  are  subsequently  fielded  the  original  requirements  are 
outdated  and  the  final  product  lags  significantly  in  current  technologies.  Any  reductions 
to  the  time  it  takes  to  deploy  these  systems  will  result  in  cost  savings  to  the  government. 
Time  reduction  is  absolutely  necessary  as  the  cost  of  new  weapons  systems  continue  an 
upward  spiral.  The  increase  in  cost,  coupled  with  the  reduction  or  redirection  in 
government  funding  for  the  DoD,  necessitates  a  reformation  of  the  system. 

The  primary  consumers  of  money  in  any  company  or  program  are  the  people  and 
the  bureaucracies  involved  with  maintaining  the  entity.  In  these  situations  cost  savings 
can  be  achieved  through  the  reduction  of  people  and  bureaucracy.  The  DT&E  and 
OT&E  entities  each  grew  up  separate  of  one  another  and  have  their  own  reporting  chains 
of  command.  While  the  separation  of  command  is  intentional,  the  redundancies  involved 
in  each  entity  can  be  eliminated.  Doing  so  would  require  reorganization  of  both 
communities. 

Accomplishing  this  reorganization  appears  to  be  the  key  to  improving  the  system. 
Attempts  have  been  made  in  the  past  by  industry  and  government.  Some  have  succeeded 
in  reducing  cost  and  schedule,  while  others  have  utterly  failed.  This  paper  will  focus  on 
one  possible  solution  to  the  problem  at  hand  through  combining  the  two  separate  test 
squadrons  into  one  test  squadron  conducting  two  missions.  A  way  to  accomplish  this  goal 
will  be  laid  out  in  an  attempt  to  better  align  to  the  realities  of  cost  and  schedule  in  the 
current  testing  environment. 
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Problem  Defined 


What  eannot  be  stated  enough  is  the  sheer  size  and  enormity  of  the  defense 
aequisition  system.  Even  at  the  lowest  levels  of  the  aequisition  proeess  the  eomplexity  of 
bureaueraey  is  mind  boggling.  Colleen  A.  Preston,  the  former  Deputy  Underseeretary  of 
Defense  for  Aequisition  Reform,  outlines  the  problem  well.  She  states: 


“The  DoD  aequisition  system  is  a  web  of  laws,  regulations,  and  polieies  adopted 
for  laudable  reasons  over  many  years.  The  intent  of  the  system  was  to  ensure 
standardized  treatment  of  eontraetors;  prevent  fraud,  waste,  and  abuse;  ensure  that 
the  government  aequisition  proeess  was  fair;  eheek  the  government's  authority 
and  its  demand  on  suppliers;  and,  enhanee  soeioeeonomie  objeetives.  While  the 
intent  of  these  provisions  is  laudable,  eombined,  the  result  is  a  eumbersome 
system  whieh  takes  too  long  to  satisfy  eustomer  requirements.  In  addition,  the 
system  plaees  administrative  burdens  on  both  DoD  and  our  suppliers  that  adds 
eost  to  the  produet  proeured.  We  ean  no  longer  afford  these  eosts  and  meet 
mission  requirements  within  eurrent  fiseal  eonstraints.”^ 


Primary  Researeh  Question 

The  primary  researeh  question  of  this  study  is:  how  ean  the  relationship  between 
developmental  and  operational  aireraft  testing  be  ehanged  to  better  align  to  the  realities  of 
eost  and  sehedule  in  the  eurrent  testing  environment? 

Subordinate  Researeh  Questions 

The  following  questions  will  assist  in  framing  the  researeh  and  help  in 
determining  a  workable  solution  to  the  eurrent  problems. 

1 .  What  is  the  eurrent  test  strueture? 

2.  What  are  the  problems  assoeiated  with  the  eurrent  test  strueture? 

3.  What  reforms  have  been  made  to  improve  the  aequisition  proeess? 
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4.  What  obstructions  have  these  reforms  encountered? 

5.  What  effeet  have  these  reforms  had  on  resolving  the  problems  associated 
with  the  eurrent  test  structure? 

6.  What  ehanges  ean  be  made  to  eliminate  these  problems? 

Significance 

Over  the  past  twenty  years  costs  associated  with  acquiring  weapons  systems  have 
sky  rocketed  (Figure  2).  During  this  same  period,  the  time  to  deploy  these  weapons 
systems  to  the  end  user  has  also  grown  tremendously.  Former  Secretary  of  Defense 
Donald  H.  Rumsfeld,  in  his  first  key  speech  on  transformation,  pointed  out  that  “it  takes 
today  twiee  as  long  as  it  did  in  1975  to  produce  a  new  weapon  system,  at  a  time  when 
new  generations  of  teehnology  are  ehumed  out  every  18  to  24  months.”"^  Due  to  more 
pressing  funding  issues  and  decreasing  budgets,  ways  must  be  found  to  reverse  these 
trends.  If  not  it  will  beeome  inereasingly  more  difficult  to  acquire  the  weapons  systems 
required  to  maintain  a  teehnological  advantage  over  our  enemies  on  the  battlefield. 
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DoD  Procurement 

(current  $  in  billions) 


Figure  2.  DoD  Procurement  Cost  History 

Source:  U.S.  Department  of  Defense.  “FY  2007  Department  of  Defense  Budget.  ”  Office  of  the 
Secretary  of  Defense.  http://www.defenselink.miFdodcmsshare/briefmgslide/l  6/060206-D- 
6570C-001.pdf  (accessed  17  March  2008):  20. 


Assumptions 


Primarily  this  study  assumes  that  there  is  a  need  to  change  the  current  system.  The 


sheer  number  of  acquisition  reform  initiatives  suggests  such  a  need.  Without  recognition 


of  the  problem,  no  progress  can  be  made  in  finding  a  solution. 
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Limitations 


This  paper  will  focus  on  testing  naval  fixed  wing  strike  fighter  aircraft  and  the 
software  and  weapons  systems  that  are  associated  with  them.  Particular  emphasis  will  be 
on  the  relationship  between  developmental  and  operational  tests  in  the  area  of  software 
and  weapons  systems  testing.  Aeromechanic al  developmental  testing  will  not  be 
discussed  in  full  detail  as  the  operational  test  relationships  are  not  as  important.  Currently 
the  Navy  has  a  separate  squadron  that  conducts  all  aeromechanical  developmental  flight 
testing  of  strike  fighter  aircraft.  Aeromechanical  testing  expands  the  envelope  in  which 
the  aircraft  operates  and  therefore  is  not  repeated  during  the  operational  test  phase. 

Summary 

This  chapter  briefly  described  the  complexity  of  the  acquisition  reform  efforts  and 
the  problem  as  related  to  the  scope  of  this  study.  The  following  chapters  will  further 
detail  the  problems  associated  with  the  current  structure  of  the  U.S.  Navy’s 
developmental  and  operational  test  community  relationships  and  a  proposed  solution  to 
the  problems  identified. 


'  U.S.  Department  of  Defense.  Defense  Acquisition  Transformation  Report  to 
Congress.  Office  of  the  Secretary  of  Defense,  Washington  D.C.:  Government  Printing 
Office,  February  2007:  2. 

Frank,  Dr  Deborah  F.  "A  Theoretical  Consideration  of  Acquisition  Reform." 
Acquisition  Review  Quarterly.  (Summer  1997):  288. 

Preston,  Colleen  A.  "Acquisition  Reform:  Making  It  A  Reality."  Acquisition 
Review  Quarterly.  (Winter  1994):  8. 
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^  Rumsfeld,  Donald  H.  DoD  Acquisition  and  Logistics  Excellence  Week  Kickoff- 
Bureaucracy  to  Battlefield.  Remarks  at  the  Pentagon.  10  September  2001. 
http://www.defenselmk.mil/speeches/speech.aspx?speechid=430  (accessed  30  January 
2008). 


9 


CHAPTER  2 


LITERATURE  REVIEW 

Introduction 

The  complexity  and  enormity  of  the  defense  aequisition  system,  eoupled  with  the 
numerous  attempts  at  reforming  the  system  have  led  to  a  vast  body  of  literature  available 
on  the  subject  of  acquisition  reform.  This  study  attempts  to  focus  the  reform  effort  on  the 
relationship  between  the  U.S.  Navy’s  strike  fighter  developmental  and  operational  test 
squadrons.  Additionally,  there  was  more  emphasis  placed  on  recent  literature,  within  the 
last  decade,  in  an  attempt  at  making  the  study  more  relevant.  Previous  literature  was 
reviewed,  and  where  still  relevant,  ineluded  in  this  study. 

The  primary  sourees  of  literature  reviewed  for  this  study  inelude  the  numerous 
articles  in  professional  journals  assoeiated  with  the  aequisition  community.  These 
journals  provide  the  most  up  to  date  information  available  on  the  subject  of  acquisition 
reform.  Additionally,  the  annual  conferenee  proceedings  held  by  the  professional 
soeieties  were  a  vast  souree  of  knowledge  and  discussion.  Numerous  books  have  been 
written  on  the  subjeet  of  aequisition  reform  but  the  constant  changes  made  to  the 
aequisition  proeess  in  recent  years  make  these  works  dated  but  they  provided  a  good 
historical  perspective  on  ehanges  already  made  and  their  results.  Lastly  a  review  of 
various  studies  conducted  by  research  companies  and  other  graduate  researeh  rounded 
out  the  material  available.  The  common  theme  throughout  the  literature  review  is  there  is 
a  need  to  change  the  current  system  to  make  it  more  cost  effective  and  more  efficient. 
Additionally,  a  reeurring  theme  of  involving  all  parties  in  the  aequisition  process 
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throughout  the  entire  life  cyele  of  a  product,  from  initial  development  to  deployment  and 
then  disposal,  was  present. 


Background 

In  order  to  fully  understand  the  problems  associated  with  the  current  structure,  a 
brief  discussion  of  the  histories  of  the  two  squadrons  involved  and  a  discussion  of  the 
current  acquisition  structure  is  required. 

Brief  History  of  Air  Test  and  Evaluation  Squadron  THREE  ONE  (VX-31) 

VX-3 1  traces  its  origins  back  to  the  formation  of  China  Lake  itself  The  Naval 
Ordnance  Test  Station  (NOTS)  was  established  8  November  1943  by  the  Secretary  of  the 
Navy  with  a  mission  to  have  “a  station  having  for  its  primary  function  the  research, 
development  and  testing  of  weapons,  and  having  the  additional  function  of  furnishing 
primary  training  in  the  use  of  such  weapons."  ^  Through  the  years  this  mission  has 
remained  roughly  unchanged  though  the  name  of  the  institution  varied  from  NOTS  to  the 
Naval  Weapons  Center  (NWC)  in  1967,  then  to  the  Naval  Air  Warfare  Center  Weapons 
Division  (NAWCWD)  on  22  January  1992.  Throughout  this  time  what  would  become 
VX-3 1  was  the  air  test  arm  of  these  institutions  and  provided  the  means  by  which  to  test 
the  weapons  and  systems  developed. 

On  8  May  1995  the  Naval  Weapons  Test  Squadron,  China  Lake  was  established 
as  a  separate  entity  from  NAWCWD  although  the  reporting  chain  of  command  remained 
the  same.  As  part  of  a  reorganization  within  the  test  community,  the  squadron  was  re¬ 
designated  Air  Test  and  Evaluation  THREE  ONE  (VX-31)  on  1  May  2002.^  VX-31 
ultimately  reports  through  NAWCWD  to  the  Naval  Air  Systems  Command  (NAVAIR)  in 
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Patuxent  River,  Maryland.  VX-Sl’s  eurrent  mission  is  to  “provide  the  resources, 
expertise  and  support  needed  to  plan  and  execute  safe  and  efficient  ground  and  flight  test 
of  developmental  weapons  and  weapon  systems  . . .  operate  a  cost  effective  and  efficient 
flying  test  bed  program  to  support  the  DoD  RDT&E  [Research,  Development,  Test,  & 
Evaluation]  community.”^  To  accomplish  this,  VX-31  maintains  19  E/A-18  Hornet 
aircraft  of  varying  models  (A-G),  along  with  other  platforms  associated  with  the  Naval 
and  Marine  Corps  aviation  community.  Supporting  the  mission  there  are  33  aircrew,  322 
maintainers  and  support  personnel,  and  over  2,000  engineers  and  contractors."^ 

Brief  History  of  Air  Test  and  Evaluation  Squadron  NINE  (VX-9) 

VX-9  traces  its  lineage  back  to  Air  Development  Squadron  EIVE  (VX-5)  which 
was  established  18  June  1951.  VX-5’s  mission  was  to  “develop  and  evaluate  aircraft 
tactics  and  techniques  for  delivery  of  airborne  special  weapons.”^  VX-5  moved  to  China 
Eake,  California  in  1956.  Through  the  years  VX-5  maintained  detachments  throughout 
the  United  States  to  evaluate  systems  in  varying  climates.  In  June  1993  the  Chief  of 
Naval  Operations  (CNO)  directed  that  the  efforts  of  VX-4  and  VX-5  be  combined  into  a 
single  evaluation  squadron  as  part  of  the  “right  sizing”  of  the  force  structure  following 
the  end  of  the  Cold  War.  On  30  April  1994  Air  Test  and  Evaluation  Squadron  NINE  was 
established  at  China  Eake,  California.^ 

VX-9’s  current  mission  is  to  operate  as  a  force  to  conduct  an  “independent  and 
objective  evaluation  of  the  operational  effectiveness  and  suitability”  of  all  platforms  and 
systems  destined  for  use  in  the  U.S.  Navy  and  to  develop  tactical  procedures  for  their 
employment.^  As  a  part  of  this  mission,  VX-9  is  assigned  to  report  the  results  of 


operational  evaluations  conducted  to  Commander  Operational  Test  and  Evaluation  Eorce 
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(COMOPTEVFOR  or  COTF).  However,  VX-9’s  administrative  chain  of  command 
reports  to  Commander  Naval  Air  Forces  Pacific  Fleet  (COMNAVAIRPAC  or  AIRPAC) 
To  accomplish  this  mission  VX-9  has  fifteen  F/A-18  Hornet  aircraft  of  varying  models 
(A-F)  along  with  other  platforms  associated  with  the  Naval  and  Marine  Corps  aviation 
community.  There  are  30  aircrew,  over  200  maintainers  and  support  personnel,  and  33 
engineers  and  contractors  directly  associated  with  VX-9  in  support  of  the  mission.^ 

Current  Test  Structure 

The  current  structure  of  the  defense  acquisition  system  is  defined  by  the 
Department  of  Defense  (DoD)  5000  series  documents.  These  documents  spell  out  in 
detail  how  to  run  the  defense  acquisition  system.  To  delve  into  detail  is  not  required  for 
the  purposes  of  this  paper.  What  is  required  is  a  general  understanding  of  how  testing  is 
structured  in  relation  to  developmental  and  operational  test.  This  basic  knowledge  will 
help  when  we  compare  options  on  how  to  reform  the  system. 

The  primary  objective  of  defense  acquisition  is  to  “acquire  quality  products  that 
satisfy  user  needs  with  measurable  improvements  to  mission  capability  and  operational 
support,  in  a  timely  manner,  and  at  a  fair  and  reasonable  price.” The  current  defense 
acquisition  management  framework  is  shown  in  Figure  3. 
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Figure  3.  Defense  Acquisition  Management  Framework 

Source:  Dillard,  J.T.,  “Toward  Centralized  Control  of  Defense  Acquisition  Programs”  Defense 
Acquisition  Review  Journal,  Volume  12  No  3  (2005):  337. 


This  model  is  used  by  DoD  in  the  acquisition  of  new  programs.  To  better  understand 
where  the  two  communities  fall  within  the  framework,  Figure  4  provides  for  a  breakdown 


of  the  system  into  area  competencies. 
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Figure  4.  Defense  Aequisition  Proeess  with  Test  and  Evaluation  Timelines 

Source:  Barnette,  Gregory  L.  "Test  and  Evaluation  in  a  Dynamie  Aequisition  Environment." 
Defense  Acquisition  Review  Journal  Volume  37.  (Deeember  -  Mareh.  2005):  338. 


Ideally  the  primary  areas  where  OT&E  has  input  into  the  aequisition  process  is  during  the 
Early  Operational  Assessment  (EOA),  Operational  Assessment  (OA),  and  during  the 
OT&E  periods  denoted.  Other  than  those  periods  OT&E  aircrew  have  been  encouraged 
to  participate  in  DT&E  as  observers.  Communication  between  DT&E  and  OT&E  is 
essential  during  these  times  as  the  developmental  process  is  ongoing. 

During  the  last  decade  acquisition  reforms  have  shown  a  need  for  a  different  way 
to  acquire  new  systems  due  to  the  costs  associated  with  current  programs.  This  need, 
coupled  with  the  move  from  a  “threat  based”  to  a  “requirements  based”  acquisition 
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strategy,  due  to  the  end  of  the  Cold  War,  have  driven  the  aequisition  community  and 
DoD  to  implement  spiral  development  of  systems.  The  spiral  development  model  is 
found  in  Figure  5. 


Source:  13  Aerospace.  "Providing  Sustained  Maritime  Surveillance  on  a  National  Scale  is  a  Tall 
Order."  http://www.i3aerospace.com/strategy  (accessed  13  June  2008). 
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Spiral  development  builds  upon  previous  work  and  institutes  new  technologies  once  they 
become  mature  enough  to  add  to  a  system.  The  reason  to  institute  a  spiral  development 
structure  is  to  help  drive  down  costs  and  decrease  the  time  to  field  new  capabilities. 
While  the  effectiveness  of  these  changes  is  yet  to  be  determined,  this  is  the  current 
system  construct  this  study  will  use  when  recommending  changes. 

Transformation 

“Transformation  is  a  process  and  a  mindset-  not  a  product.”-  Gen  Richard  B 
Myers,  USAF,  Former  Chairman  Joint  Chiefs  of  Staff" 

Transformation  has  taken  on  many  meanings  over  the  years  and  specifically  so  in 
the  acquisition  world.  Current  acquisition  reform  is  trying  to  model  commercial  industry 
and  achieve  the  successes  that  U.S.  businesses  have  had  in  the  global  marketplace 

through  maximizing  productivity.  However,  the  reforms  in  government  have  failed  to 
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achieve  the  same  successes  as  in  industry  leading  to  a  continual  parade  of  reforms. 
Annually,  new  initiatives  or  directives  are  attempted  within  the  acquisition  community. 
Many  fail  before  they  ever  fully  get  implemented.  Others  are  a  casualty  of  the  cultural 
bias  inherent  in  all  organizations. 

The  players  involved  in  transforming  the  acquisition  system  are  as  numerous  as 

the  initiatives  that  have  been  attempted  at  reforming  the  structure.  The  primary  reasons 

many  reforms  have  failed  have  been  due  to  the  sheer  size  of  institutions  and  how  many 

people  the  initiatives  affect.  Primarily  the  acquisition  system  is  controlled  by  two  major 

players.  Congress  and  the  Department  of  Defense.  While  only  two  main  institutions 

control  the  acquisition  process,  the  details  in  the  relationship  between  the  two  entities  is 

complicated.  Laws  enacted  by  Congress  put  numerous  regulations  on  DoD  as  to  how 
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acquisitions  should  be  accomplished.  These  laws  have  ereated  numerous  departments 
within  DoD  that  eomplieate  the  proeess.  The  Deputy  Seeretary  of  Defense  for 
Aequisitions  controls  the  proeurement  and  test  activity  while  the  Director  of  Operational 
Test  and  Evaluation  (DOT&E)  eontrols  oversight  of  the  programs  for  operational 
purposes.  These  two  eoupled  with  numerous  other  agencies  to  monitor  joint  testing,  joint 
aequisitions,  and  other  aspeets  assoeiated  with  the  defense  aequisition  system  provide  for 
an  enormous  bureaueracy  that  is  diffieult  to  eontrol  or  reform.  Before  we  reform  we  must 
have  a  reason  for  reforming,  and  understand  the  impediments  that  ean  oecur  when 
attempting  to  ehange  a  large,  bureaueratie  system. 

Why  Reform? 

Aequisition  reform  should  make  the  system  more  effieient,  improve  business 

praetiees,  and  allow  buying  more  with  less.  Reeall  the  publie  outrage  in  the  late  1980’s 

when  the  $400  hammer  stories  broke  in  the  press.  The  publie  was  outraged  at  what 

appeared  to  be  irresponsible  spending  and  that  alone  remains  a  strong  argument  for 

reforming  the  acquisition  system.  Elntil  2002,  DoD  had  eondueted  128  studies  on 

aequisition  reform  and  identified  a  slow  bureaueratie  proeess  as  the  main  impediment  to 

ehange.  This  is  the  same  problem  that  subsequent  eommissions  indieate  are  still  present 

within  the  aequisition  eommunity  today. This  suggests  that  no  reforms  were  seriously 

attempted  by  the  organizations  that  were  identified  or  that  the  initiatives  attempted  failed 

in  their  purpose.  The  failure  further  suggests  an  organizational  eulture  resistant  to  change. 

This  culture  is  manifested  in  the  bureaucraey  and  status  quo  of  the  eurrent  system.  The 

“we’ve  never  done  it  that  way  before”  attitude  is  pervasive  in  the  eurrent  system.  In  order 

for  true  change  to  oeeur  within  institutions,  obstruetions  to  ehange  require  reeognition 
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and  tailored  efforts  must  be  made  to  speeifioally  target  these  eultural  and  institutional 
obstruetions.  Finally  the  motivation  to  ehange  is  laeking,  simply  beeause  the  eurrent 
systems  works  however  eostly  and  ineffieient.'^ 

The  greatest  reasons  for  reform  were  spelled  out  reeently  in  the  International  Test 
and  Evaluation  Assoeiation  (ITEA)  Journal  artiele  by  Brian  M.  Simmons: 

“(1)  The  systems  aequisition  and  assoeiated  T&E  proeess  has  ehanged  during 
wartime  and  is  unlikely  to  return  to  the  traditional  proeess  when  the  war  is  over; 

(2)  Business  transformation  demands  more  efficieney  in  T&E  processes  now;  and 

(3)  Networked  testing  requires  the  testers  to  rely  on  each  other  as  opposed  to 
staying  in  traditional  Service,  local  test  range  or  developmental  test 
(DT)/operational  test  (OT)  domain.”'^ 

The  OT&E  community  has  realized  these  imperatives  and  has  started  to  voice  their 
concern.  Thomas  P.  Christie,  a  former  DOT&E,  put  it  this  way,  “Either  we  change  our 
way  [OT&E]  of  doing  business,  adapt  to  the  new  acquisition  paradigms  and  the  war  on 
terrorism,  or  we  will  find  ourselves  becoming  irrelevant.”'^ 

Resistance  to  Change  and  Its  Affect  on  Reform 

“The  only  people  who  like  change  are  wet  babies.”  -  Sir  Brian  Wolfson 

No  reformation  process  occurs  easily.  There  are  always  obstacles  to  change. 
Critics  of  reform  point  to  its  past  failings  and  political  slant  rather  than  become  an  agent 
for  change.  The  obstacles  are  generally  the  same.  The  greatest  obstacles  to  change  are 
the  organizations  themselves.  No  one  likes  learning  to  do  new  things.  People  become 
accustomed  to  how  things  are  and  change  causes  discomfort.  If  change  is  considered  a 
threat  to  the  organizational  rule  it  is  fought  at  all  levels  within  the  institution.  The  larger 
the  organization,  the  more  difficult  it  is  to  change.  However,  in  order  to  remain 
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competitive,  businesses  will  typieally  institute  ehange  when  market  forees  dietate.  In  so 
doing  they  reeognize  areas  needing  ehange  and  institute  those  ehanges  in  a  timely 
manner.  Why  then  is  it  so  hard  for  the  government  to  ehange  its  way  of  doing  business? 
Speeifieally,  why  ean  the  OT&E  eommunity  not  ehange  to  improve  effieieney  and  reduee 
eost?  Talking  about  why  transformation  objeetives  have  failed  in  the  past,  Mr.  Christie 
stated  "We've  [OT&E]  never  really  taken  [these  reform  efforts]  to  heart;  granted,  we  ... 
eontinue  to  make  ehanges  to  the  proeess,  but  we  have  yet  to  really  eome  to  grips  with 
some  of  the  root  eauses." 

Change  ean  and  does  work  as  evident  in  the  business  eommunity.  In  order  to 
aehieve  lasting  ehange,  institutional  stove-pipes  must  be  eliminated.  Change  must  be 
driven  by  superiors.  However,  real  eost  savings  and  streamlining  eannot  be  done  at 
higher  levels  but  must  be  aeeomplished  only  by  the  members  of  eaeh  partieipating 
ageney.^"^  The  key  is  to  remember  transformation  is  not  a  plaee  in  time  or  the  delivery  of 
a  eapability.  Rather,  it  is  a  eommitment  to  sensible,  event-driven,  proaetive  ehange.  It  is  a 
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journey,  not  a  destination. 

Cost  &  Sehedule 

During  the  mid  I980’s,  Norman  Augustine  wrote  a  humorous  eharaeterization  of 
the  eost  growth  of  military  aireraft.  His  Eaw  XVI  states: 

“In  the  year  2054,  the  entire  defense  budget  will  purehase  just  one  aireraft.  This 
aireraft  will  have  to  be  shared  by  the  Air  Eoree  and  Navy  3-1/2  days  eaeh  per 
week  exeept  for  leap  year,  when  it  will  be  made  available  to  the  Marines  for  the 
extra  day.”^^ 
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While  this  may  be  humorous,  the  growth  of  weapons  system  eost  seems  to  lead  to  this 
eventuality.  Cost  and  sehedule  drive  every  major  aequisition  program.  Deeisions  that  are 
made  are  usually  done  solely  on  these  two  prineiples.  Figure  6  and  Figure  7  show  the 
aireraft  program  trends  over  the  past  50  years. 


Figure  6.  Aircraft  Program  Cost 

Source:  Eskew,  Dr.  Flenry  L.  "Aircraft  Cost  Growth  and  Development  Program  Length:  Some 
Augustinian  Propositions  Revisited."  Acquisition  Review  Quarterly.  (Summer  2000):  211. 


Aircraft  cost  over  the  period  analyzed  by  Figure  6  grew  at  an  inflation  adjusted 
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rate  of  3%  per  year.  This  is  of  modest  growth.  However,  the  complex  systems  in  the 
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acquisition  process  today  (F-22,  V-22,  and  Joint  Strike  Fighter  aireraft),  eoupled  with  the 
probability  of  deereased  buys  will  likely  drive  this  growth  rate  substantially  higher. 


Figure  7.  Aircraft  Development  Program  Length 

Source:  Eskew,  Dr.  Henry  L.  "Aircraft  Cost  Growth  and  Development  Program  Length:  Some 
Augustinian  Propositions  Revisited."  Acquisition  Review  Quarterly.  (Summer  2000):  215. 


While  seemingly  independent  of  one  another,  the  two  variables  of  eost  and 
sehedule  are  intertwined  and  eonstitute  the  majority  of  the  problems  faeed  with  any 
program  development.  Figure  7  shows  that  as  program  length  inereases  so  does  eost.  It  is 
estimated  that  a  two  month  inerease  in  program  length  roughly  equates  to  $1  million  in 
1990  adjusted  eost.  Over  the  past  20  years,  the  time  to  deploy  new  weapons  systems  or 
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capabilities  has  increased  dramatieally.  Any  reduetion  in  the  time  to  deploy  these  systems 
direetly  relates  to  a  decrease  in  cost  for  the  overall  program.  In  aequisition,  time  is 
money. 

Department  of  Defense  funding  has  been  in  decline  throughout  mueh  of  the  past 
20  years.  Figure  8  shows  the  historical  relationship  and  trend  of  the  DoD  budget  since  the 
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end  of  World  War  II.  The  general  trend  is  expected  to  eontinue  to  decline. 


DoD  Budget  as  %  of  GDP 
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Figure  8.  Defense  Budget  as  a  Percent  of  GDP 

Source:  U.S.  Department  of  Defense.  “FY  2007  Department  of  Defense  Budget.  ”  Office  of  the 
Secretary  of  Defense.  http://www.defenselink.miFdodcmsshare/briefingslide/l  6/060206-D- 
6570C-001.pdf  (accessed  17  March  2008):  26. 
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The  budgets  assoeiated  with  aequisitions  of  new  systems  and  updates  to  existing  systems 
have  mirrored  this  deeline  (Figure  9).  Coupled  with  the  deeline  in  funding  is  the  increase 
in  costs  associated  with  acquiring  new  state-of-the-art  equipment  (see  Figure  2).  The  two 
funding  crunches  have  often  meant  reduction  of  testing  to  save  money.  The  realities  of 
reduced  budgets  and  increasing  costs  do  not  appear  to  be  reversing  anytime  soon. 
Therefore,  solutions  must  be  implemented  by  industry  and  government  to  reduce  cost  and 
increase  efficiencies  during  the  acquisition  life  cycle. 


Figure  9.  DoD  Budget  Trends 

Source:  U.S.  Department  of  Defense.  FY 1997  Annual  Report.  Director  Operational  Test  & 
Evaluation.  Washington  D.C.:  Government  Printing  Office,  1997. 
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Cost  and  schedule  have  been  the  target  of  nearly  every  major  aequisition  reform 
initiative  over  the  past  50  years.  The  Global  War  on  Terrorism  has  inereased  demand  for 
newer  and  better  systems  or  eapabilities.  The  customer  wants  and  needs  these  systems 
quiekly  whieh  inereases  the  potential  of  eireumventing  a  rigorous  and  lengthy  aequisition 
proeess.^^  Any  solution  that  would  enable  eontrolling  eost  and  sehedule  or  minimizing 
the  impact  to  programs  would  greatly  enhanee  the  test  eommunity. 

Communieation 

Communieation  between  the  DT&E  and  OT&E  squadrons  at  China  Eake  is 
dysfunctional.  A  recent  2007  survey  eondueted  by  the  National  Defense  Industrial 
Assoeiation  (NDIA)  showed  that  a  majority  of  their  2007  national  eonvention 
participants  thought  the  relationship  between  the  DT&E  and  OT&E  eommunities  was 
“strained,  eonflieting,  minimal,  sometimes  non-existent,  somewhat  hostile.”  The  laek  of 
eommunieation  results  in  three  very  serious  problems;  no  OT  early  involvement, 
redundancy  in  testing,  and  operational  evaluators  not  familiar  with  the  systems  they  are 
evaluating.  The  absenee  of  eommunieation  stems  from  the  inherent  strueture  of  the  two 
squadrons  whieh  have  no  eommon  ehain  of  eommand.  VX-9  answers  to  COTE  in 
Norfolk,  Virginia,  while  VX-31  is  a  subsidiary  of  Naval  Air  Systems  Command 
(NAVAIR)  in  Patuxent  River,  Maryland.  Often  any  program  eommunieation  between 
VX-9  and  VX-3 1 ,  loeated  physieally  200  yards  apart,  must  be  routed  through  entities 
residing  on  the  east  eoast  thousands  of  miles  away.  Due  to  this,  eommunieation  between 
DT&E  and  OT&E  is  usually  delayed  weeks  at  a  time  as  offieial  messages  must  get 
written  for  formal  eommunieation. 
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The  failure  of  eommunieations  is  not  just  a  problem  at  China  Lake  but  rather  a 
systemie  problem  of  the  strueture  of  the  two  organizations.  In  testimony  to  Congress  on 
the  V-22  program,  Program  Manager  Colonel  Nolan  Schmidt,  USMC,  stated  the  "lack  of 
effective  communications  ...  was  an  outgrowth  of  deliberately  separate  test  and  program 
office  structures."  He  further  charged  that  "contact  between  OPEVAL  and  the  program 
office  is  a  deviation  from  the  norm.”^^  The  delays  and  current  structure  are  unnecessary 
and  an  enormous  barrier  to  communication. 

Personnel  Management 

Personnel  associated  with  the  acquisition  system  are  another  potential  problem 
area.  People  are  huge  consumers  of  money  in  any  business,  and  the  acquisition 
community  is  no  exception.  In  testimony  before  the  House  National  Security  Committee 
the  then  Director  of  Test,  Systems  Engineering  &  Evaluation  Dr  Patricia  Sanders  stated; 

“It  is  clear  that  the  bulk  of  our  money  is  spent  on  manpower-at  least  54%  for 
government  personnel  and  an  additional  15%  for  contractor  services. .  .if  a  reinvention 
idea  does  not  reduce  required  manpower,  it  will  not  have  a  significant  impact.”^^ 

Recent  reform  initiatives  have  attempted  to  address  the  staffing  concerns  within 
the  acquisition  community.  Eigure  10  shows  the  reduction  in  acquisition  staffing  as  a 
result  of  the  reform  initiatives. 
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Figure  10.  Acquisition  Workforce  Staffing  Levels 

Source:  U.S.  Department  of  Defense.  FY  2001  Annual  Report.  Director  Operational  Test  & 
Evaluation.  Washington  D.C.:  Government  Printing  Office,  2001. 


The  reduction  in  staffing  is  seen  as  an  attempt  at  reducing  the  duplication  of  effort  within 
the  current  structure.  Personnel  management  was  a  primary  transformation  initiative  of 
Secretary  Rumsfeld  as  he  saw  the  need  to  ask  tough  questions  on  redundant  staffs. 
Slashing  duplication  would  make  decisions  move  quicker  and  encourage  more 
cooperation.^"^  While  these  cuts  have  gone  far,  greater  efficiencies  could  be  made. 
Additionally  the  Navy  is  already  having  difficulties  fully  staffing  both  squadrons  with  the 
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personnel  “required”  to  get  the  job  done.  Currently,  aircrew  staffing  at  VX-9  is  at  74% 
and  VX-3 1  is  at  63%  of  the  required  levels.  Both  squadrons  have  been  operating  at  a 
reduced  level  for  years  and  the  job  is  still  getting  done.  Therefore,  efficiencies  could  be 
made  in  the  area  of  personnel  management. 
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Test  Structure 


The  test  structure  associated  with  the  two  squadrons  is  decades  old.  While  there 
have  been  some  changes  made  to  the  structure,  no  meaningful  benefit  has  resulted  from 
these  changes.  What  is  needed  is  a  comprehensive  review  of  how  to  accomplish  testing  in 
today’s  acquisition  environment.  The  Revolution  in  Military  Affairs  (RMA)  that  has 
evolved  over  the  past  decade  must  be  accompanied  with  a  Revolution  in  Business  Affairs 
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(RBA)  to  fully  transform  the  acquisition  system.  New  mindsets  and  ways  of  thinking 
must  be  generated  throughout  the  acquisition  system  in  order  to  make  any  real  reforms 
successful. 


Paths  Toward  Change 

The  path  towards  integrating  DT&E  and  OT&E  has  been  a  gradual  progression 
since  the  1960’s.  Each  subsequent  acquisition  reform  or  attempt  to  save  money  in  the 
T&E  process  enacts  new  ideas  on  how  to  move  DT&E  and  OT&E  closer  together.  The 
newest  of  these  ideas  have  been  the  processes  of  Integrated  Test  and  Evaluation  (IT&E) 
and  Joint  Test  and  Evaluation  (JT&E).  Both  of  these  processes  have  attempted  to 
improve  the  current  structure  for  the  same  reasons  mentioned  here,  but  have  failed  due  to 
the  same  problems  mentioned  also. 

Two  programs  currently  operating  at  China  Take  offer  a  view  of  these  initiatives, 
but  for  separate  reasons.  The  two  programs  are  the  current  testing  being  conducted  on  the 
EA-18G  Growler  and  the  AV-8B  Harrier  programs. 
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EA-18G  Growler 


The  EA-18G  Growler  program  is  a  testament  to  the  IT&E  proeess.  Integrated 
Product  Teams  (IPT)  enhance  communication  between  the  Navy  and  the  contractor. 
Additionally,  OT&E  has  been  brought  into  the  IPT’s  and  have  worked  side  by  side  with 
DT&E  personnel.  Both  aircrew  conduct  developmental  and  operational  testing  and 
results  are  shared  across  the  spectrum.  This  structure  has  enabled  the  program  to  achieve 
improved  results  in  cost  and  schedule  performance.  However,  the  Growler  has  yet  to 
enter  the  OT&E  phase  of  testing  where  this  interaction  should  pay  off  enormously.  The 
success  of  the  program  remains  to  be  seen.  To  be  fully  successful  this  interaction  must 
continue  until  the  last  Growler  is  in  service,  not  until  the  first  one  reaches  the  Elect  which 
is  how  it  normally  occurs. 

AV-8B  Harrier 

The  AV-8B  Harrier  program  is  nearing  the  end  of  its  life  cycle.  By  2015  the 
Harrier  will  be  replaced  with  the  new  Joint  Strike  Tighter  (JSE).  The  example  set  by  the 
Harrier  program  is  not  so  much  the  interaction  of  the  aircrews  but  rather  the  structure  of 
the  maintenance  force.  Since  airframes  are  becoming  scarce  and  resources  have 
diminished,  the  Harrier  program  was  integrated  in  2007  into  VX-3 1  ’s  maintenance 
department.  The  testing  structure  still  remains  the  same  but  now  all  aircraft  are  assigned 
to  VX-3 1 .  This  maintenance  structure  could  be  a  key  component  on  how  to  integrate  the 
two  squadrons.  Any  future  analysis  must  look  at  lessons  learned  from  this  integration. 
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Air  Force  Programs 

The  Air  Force  has  recognized  the  need  to  reform  its  processes  as  well.  In  2004  the 
Air  Force  combined  their  old  manuals  Developmental  Test  and  Evaluation  (AFI  99-101), 
Operational  Test  and  Evaluation  (AFI  99-102),  and  Live  Eire  Test  and  Evaluation  (AFI 
99-105)  into  one  comprehensive  manual.  Capabilities  Based  Test  and  Evaluation  (AFI 
99-103).  This  effort  was  undertaken  to  foster  “an  integrated  testing  philosophy  in  an 
effort  to  streamline  Test  and  Evaluation.”  AFI  99-103  espouses  an  integrated  test 
approach  and  seeks  to  employ  combined  DT&E  and  OT&E  whenever  practical.  This 
integration  will  eliminate  duplicate  testing,  correct  schedule  deficiencies,  and  identify 
program  concerns  early  in  the  process,  much  sooner  than  would  be  done  under  the 
traditional  systems.  The  end  result  will  be  a  better  product,  developed  efficiently  and 
delivered  more  quickly  to  the  Elect. 

Recently  the  Air  Eorce  has  called  for  more  changes  in  their  DT&E  and  OT&E 
relations.  In  September  of  2006  the  Air  Eorce  held  a  Rapid  Improvement  Event  (RIE)  as 
part  of  the  service’s  Air  Eorce  Smart  Operations  for  the  2Ist  Century  (AES02I)  program. 
The  RIE  was  titled  “Early  Tester  Involvement”  and  discussed  ways  in  which  early 
involvement  could  reduce  overall  weapons  system  cost.  One  of  the  conclusions  from  this 
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conference  was  the  need  for  “integrated  developmental  testing/operational  testing.” 

In  all  the  Air  Eorce  has  found  success  in  their  attempts  at  transformation.  They  are 
closer  to  succeeding  where  others  have  failed.  Acquisition  reform  does  work.  The  Air 
Eorce  saved  $20  billion  on  their  reform  initiative  at  the  turn  of  the  century."^*’  Continued 
reforms  offer  even  greater  savings.  Hopefully  instituting  the  solutions  mentioned  here 
will  enable  the  Navy  to  start  seeing  results  similar  to  these. 
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Summary 

Chapter  Two  has  laid  the  foundation  of  knowledge  needed  to  understand  the 


historical  background  of  the  organizations  and  players  involved,  a  basic  understanding  of 
the  current  acquisition  structure,  the  problems  associated  with  that  current  structure,  and 
some  programs  that  have  begun  to  reform  the  current  system.  This  basis  of  knowledge 
will  be  used  in  the  following  chapters  to  further  analyze  ways  to  fix  the  problems  of  cost, 
schedule,  communication,  personnel  management,  and  test  conduct. 
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CHAPTER  3 


RESEARCH  METHODOEOGY 

This  research  analyzes  the  current  relationship  between  the  U.S.  Navy’s 
developmental  and  operational  test  squadrons  located  atNAWC  China  Take,  California. 
In  the  previous  chapters  the  historical  setting,  current  structure,  and  problems  were 
detailed.  Chapter  Three  will  explain  the  methods  used  to  gather  the  data  presented  in  the 
following  chapters. 

The  research  conducted  was  a  comparison  of  the  available  literature  written  on  the 
subject.  No  statistical  or  mathematical  methods  were  used.  In  addition  to  available 
literature,  interviews  were  conducted  with  members  of  the  two  squadrons  along  with 
reporting  data  for  those  squadrons.  Once  the  research  was  completed  the  data  was 
compiled  for  comparison  in  this  study. 

The  first  step  in  conducting  the  research  was  to  determine  the  historical 
relationship  between  the  two  squadrons.  Through  researching  squadron  histories  the 
individual  squadron  relationships  were  easily  determined.  However,  the  historical 
interaction  between  these  squadrons  could  not  be  determined,  but  was  inferred  through 
the  literature  reviewed.  The  historical  relationship  also  included  research  into  the 
congressional  laws  governing  the  current  system.  Understanding  the  limit  of  these  laws  is 
essential  to  determining  solutions  to  any  problems  and  the  path  that  must  be  taken  to 
correct  deficiencies.  Easily,  a  historical  review  was  conducted  of  the  acquisition  reform 
initiatives  undertaken  in  the  past  50  years.  The  focus  of  this  research  was  on  what  was 
attempted,  what  worked,  what  failed,  and  why. 
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After  determining  the  historieal  baekground,  research  was  conducted  on  the 
current  structure  of  the  acquisition  process.  Understanding  the  current  acquisition  process 
is  fundamental  to  finding  problems  and  solutions. 

Once  the  current  acquisition  process  was  thoroughly  reviewed,  research  focused 
on  where  the  problem  areas  were.  This  research  consisted  of  problems  identified  through 
the  literature  review  as  well  as  problems  identified  through  interviews  and  personal 
experience  working  on  numerous  acquisition  programs.  Results  were  then  grouped  into 
categories  for  further  analysis. 

The  final  step  in  the  research  was  to  examine  the  problems  identified  and  provide 
a  possible  solution  as  to  how  the  system  could  be  improved.  The  analysis  used  for  the 
formulation  of  the  proposed  solution  was  a  compilation  of  current  programs  that  have 
been  used  recently  and  the  next  logical  step  in  the  progression  that  had  been  seen  in  the 
research.  First  the  options  available  to  reform  were  examined.  From  the  options,  criteria 
noted  from  the  research  of  communication,  cost,  schedule,  and  personnel  were  used  to 
formulate  the  best  option  to  proceed  forward  with.  This  option  was  then  vetted  in  detail 
using  the  same  criteria.  Then  a  way  to  accomplish  a  test  using  the  new  system  was 
examined  for  clarity. 

This  study  is  intended  to  offer  a  path  forward  to  solve  the  current  problems.  It 
provides  a  possible  solution  and  a  starting  point  for  further  analysis. 
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CHAPTER  4 


ANALYSIS 

Introduction 

The  previous  ehapters  have  outlined  the  historieal  setting  and  have  described  in 
depth  the  issues  assoeiated  with  the  relationship  between  the  developmental  and 
operational  test  eommunities.  Understanding  this  relationship,  to  include  the  factors  of 
cost,  schedule,  communication,  personnel  management,  and  the  conduct  of  testing  is 
eritieal  to  developing  any  solution. 

The  literature  review  (Chapter  Two)  examined  five  of  the  six  subordinate 
questions  listed  in  Chapter  One.  This  chapter  will  consider  the  final  question:  What 
ehanges  can  be  made  to  eliminate  these  problems? 

Available  Options  for  Change 

There  are  fundamentally  three  solutions  available  to  transform  the  conduct  of 
testing  strike  fighter  aireraft  within  the  U.S.  Navy.  These  solutions  are:  do  nothing  and 
try  to  work  within  the  eurrent  system,  eliminate  the  operational  test  squadron,  or  eombine 
the  two  squadrons  into  a  single  entity  to  perform  both  missions.  These  options  will  be 
evaluated  against  the  main  eriteria  noted  in  this  study  of  cost,  schedule,  communication, 
and  personnel. 

Maintain  the  Current  Strueture 

The  first  option  of  doing  nothing  is  basically  the  route  the  Navy  has  taken  in  past 
reform  efforts.  Attempting  to  work  within  the  current  system  construct  leaves  little  room 
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for  improvement.  Any  improvements  implemented  may  produee  minimal  results  because 
the  underlying  problems  still  remain. 

Cost  and  schedule  reductions  are  listed  as  one  of  the  major  factors  affecting  the 

current  T&E  structure.  *  Most  reforms  attempted  to  date  have  been  at  trying  to  achieve 

results  within  these  two  areas.  However,  progress  implementing  prior  solutions  to 

problems  has  been  limited.  Without  a  drastic  change  in  how  the  system  is  structured  or 

how  testing  is  performed,  there  is  only  a  maximum  amount  of  improvement  to  be  gained 

in  the  current  structure.  Additionally,  the  decision  cycle  time  required  to  deliver  products 

to  the  Fleet  user  has  been  reduced  beyond  the  ability  of  the  current  system  to  function 
•1 

effectively. 

Communication  under  the  current  structure  would  require  significant 
improvement  from  its  current  state.  The  stove-pipe  mindset  of  organizations  was  another 
challenge  facing  the  T&E  community."^  Figure  1 1  shows  the  current  organizational 
relationship  between  the  DT&E  and  OT&E  communities  within  DoD. 
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Figure  11.  DoD  T&E  Organizational  Chart 


Source:  National  Research  Council.  Statistical  Methods  for  Testing  and  Evaluating  Defense 
Systems:  Interim  Report.  Washington  D.C.:  National  Academy  Press,  1995.  75. 


Highlighted  in  Figure  1 1,  the  lowest  level  of  official  coordination  between  DT&E 
and  OT&E  within  the  Navy  is  located  at  the  service  chief  level.  The  lack  of  a  defined 
relationship  below  this  level  leads  to  numerous  problems  within  the  current  system. 
Attempts  at  improving  communication  within  this  structure  have  lead  to  integrated  test 
efforts  that  have  attempted  to  combine  contractor  test  (CT),  with  DT&E  and  OT&E. 

Even  with  these  advances,  efforts  were  still  stove-piped.^  Within  these  stove-pipes  exist 
distrust,^  competing  requirements,  and  enormous  bureaucracy^  that  effect  a  multitude  of 
areas  within  the  current  structure. 
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Staffing  levels  will  be  difficult  to  affect  with  no  change  to  the  current  structure. 
While  there  may  be  small  improvements  in  increasing  efficiency  and  eliminating 
redundancy,  no  significant  change  can  be  made  to  affect  progress  in  acquisition  reform. 

The  Navy  cannot  afford  to  continue  to  do  business  as  usual  and  more  drastic 
measures  are  required  to  realize  any  cost  savings  or  efficiencies  to  the  current  structure. 
The  current  T&E  model  is  questionable  in  a  transformational  environment  for  the  reasons 
shown.*  Due  to  these  problems  maintaining  the  current  structure  is  not  a  viable  option. 

Eliminate  OT&E 

The  reduction  in  cost  and  schedule  that  could  be  achieved  through  the  elimination 
of  OT&E  would  be  minimal  to  the  overall  program  cost  and  schedule.^  Where  the 
savings  would  be  found  is  in  the  overhead  associated  with  the  OT&E  squadron  and 
personnel  assigned.  There  would  also  not  be  a  communication  problem  as  there  would  be 
no  requirement  to  communicate  with  an  entity  that  does  not  exist. 

Eliminating  the  operational  test  squadron  initially  sounds  like  the  best  option,  and 
the  one  with  the  greatest  cost  savings.  However,  in  order  to  achieve  these  cost  savings 
some  tradeoffs  will  have  to  be  made.  There  are  fundamental  differences  in  how  and  what 
DT&E  and  OT&E  test.  Eigure  12  shows  the  testing  differences  between  the  two 
communities. 
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1  How  Tests  are  Conducted  | 

DT&E  testing  is  properly  conducted: 

•  In  a  controlled  environment  that 
minimizes  the  chance  that  unknown 
or  unmeasured  variables  will  affect 
system  performance 

•  By  technical  personnel  skilled  at 
“tweakina"  to  maximize 
performance 

•  Against  simulated  threats  tailored 
to  demonstrate  various  aspects  of 
specified  system  technical 
performance 

OT&E  testing  is  properly  conducted: 

•  In  an  operationally  realistic  environment  (e.g., 
high  seas,  temperature  extremes,  high  density 
electromagnetic  environments)  under  conditions 
simulating  combat  stress  and  peacetime 
conditions 

•  With  Fleet  operators  and  maintenance  personnel 

•  Against  threats  which  replicate,  as  closely  as 
possible,  the  spectrum  of  operational  characters 

•  Using  Fleet  tactics 

1  Testing  Subject/Topic  I 

DT&E  tests  a  weapon  or  a  "blackbox," 
whatever  the  development  program 
involves.  (Seldom  does  a  development 
program  involve  a  complete  weapon 
system.) 

OT&E  tests  total  weaoon  svstems.  If  a  missile  is  beina 
developed,  OT&E  does  not  test  only  the  missile  itself, 
but  rather  the  missile  svstem.  which  includes  the  firina 
platform;  that  platform's  detection,  classification,  and 
targeting  systems;  the  people  who  man  it;  logistical 
support;  interfacing  equipment;  etc. 

1  Evaluation  Criteria  I 

DT&E  -  Technical  criteria  are  parameters 
measured  during  controlled  DT&E  tests. 

OT&E  -  Operational  criteria  are  the  CNO-provided 
minimum  acceptable  operational  performance 
requirements  (older  programs)  or  measures  of 
effectiveness/suitability  (newer  programs),  or  thresholds, 
which  quantify  the  Critical  Operational  Issues  (COI). 

1  Measurement  and  Frequency  | 

DT&E 

•  The  DA  generally  knows  what 
he/she  wants  to  measure  (some 
particular  parameter:  launch 
velocity;  the  number  of  g’s  pulled 
as  the  missile  acquires:  time  to 
climb;  etc  ). 

•  DT&E  tests  are  structured  to  hold 
many  things  constant,  isolate 
others,  and  allow  measurement  of 
one  or  two  parameters  of  interest. 

•  It  is  generally  possible  to  verify  data 
statistically  through  replication  of 
tests. 

OT&E 

•  It  is  often  not  possible  to  specify  measurements. 

•  The  objective  is  often  simply  to  create  combat 
conditions  as  closely  as  possible  and  record  data 
as  events  unfold. 

•  For  aviation  OT&E,  with  highly  time-compressed 
test  events  and  a  high  cost  for  OT&E,  it  is 
mandatory  that  OTDs  know  exactly  what 
parameters  of  their  system  must  be  examined  to 
resolve  the  specified  COI . 

•  OT&E  cannot  enjoy  the  luxury  of  isolating  a 
variable.  Methods  of  data  capture  must  be 
devised  during  operational  evolutions  or  during 
postoperation  analysis. 

•  It  is  often  not  possible  to  replicate  data  because 
interactions  during  tests  are  unique. 

Figure  12.  DT&E  and  OT&E  Test  Comparison 


Source:  Commander,  Operational  Test  &  Evaluation  Force.  “Operational  Test  Directors  Manual.” 
COMOPTEVFORINST  3980.1.  Norfolk,  Virginia.  April  2008:  2-6,  2-7. 
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Eliminating  testing  requires  a  balaneed  approach  with  risk.  How  much  risk  is  the 
government  willing  to  accept  with  a  given  system.  There  is  never  enough  time  or  money 
to  achieve  absolute  certainty  in  testing  so  a  balanced  risk  assessment  must  be  made. 
Eliminating  duplicate  testing  poses  little  risk.  Additionally,  with  the  reduced  decision 
cycle  time,  the  ability  of  the  operational  tester  to  provide  timely,  value-added  information 
in  the  current  structure  has  been  removed.  Yet  this  must  be  balanced  with  what  tests 
would  not  be  performed,  and  what  criteria  systems  would  be  evaluated  against. 

Research  has  shown  that  operational  test  is  needed  and  provides  a  different 
perspective  on  system  performance.  The  picture  of  the  overall  system  is  invaluable  in 
providing  a  quality  product  to  the  end  user.  Additionally,  operational  test  is  mandated  by 
Congress  ,  so  eliminating  the  operational  test  squadron  cannot  be  easily  accomplished. 
The  downsides  to  eliminating  OT&E  far  outweigh  any  benefits.  Therefore,  eliminating 
OT&E  is  not  a  viable  option. 

Combine  DT&E  and  QT&E 

The  final  option  is  to  combine  the  developmental  and  operational  test  squadrons. 
Numerous  studies  have  put  forth  this  option  as  solving  the  major  ills  within  the 
acquisition  community.  A  2005  COMOPTEVEOR  report  conducted  under  guidance  from 
the  Chief  of  Naval  Operations  (CNO)  concluded; 

“The  use  of  combined  DT/OT  has  been  identified  as  one  of  the  most  prevalent 
and  successful  ‘innovative’  approaches  to  the  planning  and  conduct  of  T&E  ... 
[and]  integrated  test  should  be  viewed  as  an  order  of  magnitude  increase  in 

1  T 

collaboration  over  what  occurs  currently  in  combined  DT/OT.” 
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DoD  continues  to  move  toward  this  path  of  integration.  As  reeently  as  December 
2007,  DOT&E  Dr  Charles  MeQueary  stated  in  a  memorandum  “developmental  and 
operational  test  aetivities  shall  be  integrated  and  seamless  throughout  the  system  life 
eyele.”^"^  The  best  way  to  do  this  is  to  eombine  the  squadrons.  The  following  diseusses 
the  merits  of  this  option  and  one  solution  on  implementing  this  change. 

Proposed  Solution 

Prior  to  putting  forth  any  solution,  the  players  involved  must  be  identified  since 
they  will  affeet  any  solution  developed.  As  stated  previously  Congress  wields  enormous 
power  within  the  defense  aequisition  system.  The  laws  enacted  drive  the  framework  of 
the  DoD  5000  series  doeuments.  Both  Congress  and  the  DoD,  ineluding  the  various  sub 
ageneies  that  eneompass  the  aequisition  eommunity  and  the  operational  test  community, 
will  have  tremendous  power  in  determining  if  the  final  proposed  solution  will  work. 

The  proposed  solution  to  the  problems  defined  earlier  and  numerous  other  issues 
assoeiated  with  the  relationship  between  DT&E  and  OT&E  is  to  eombine  the  two 
squadrons  into  a  single  test  entity  under  a  single  chain  of  command.  The  goal  of 
eombining  the  two  squadrons  would  be  to  reduee  overhead,  improve  management, 
eliminate  duplieation,  and  tighten  eontrol  to  minimize  eost  growth  and  sehedule 
slippage. 

The  question  of  eombining  T&E  assets  and  struetures  is  not  a  new  one.  Usually 

the  talk  is  of  a  single  DoD  T&E  organization.  This  paper  will  not  take  it  to  that  level,  but 

instead  will  suggest  a  single  Navy  T&E  structure.  The  pursuit  of  greater  eost 

effeetiveness  and  effieieney  leads  to  this  eourse  of  aetion.'^  During  his  eonfirmation 

hearing  before  Congress,  current  Seeretary  of  Defense  Robert  M.  Gates  outlined  as  one 
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of  his  key  tenets  the  need  to  “balance  the  acquisition  and  operational  testing  processes 
between  reducing  costs  and  accelerating  schedules.”^’  While  this  solution  may  not  be 
what  he  had  in  mind,  it  certainly  addresses  his  required  outcome. 

The  biggest  speed  bump  on  the  road  to  true  joint  testing  lies  between  the  DT&E 
and  OT&E  communities.  The  focus  of  the  remainder  of  the  discussion  will  be  on  how 
to  combine  the  two  squadrons,  what  the  new  structure  will  look  like,  and  analyze  the 
efficiencies  gained  through  the  new  relationship.  Einally,  a  simulated  test  conduct  will  be 
discussed  to  better  understand  how  to  implement  this  new  process.  It  must  be  understood 
that  change  is  neither  easy  nor  comfortable  for  those  involved.  The  drastic  changes  that 
have  occurred  in  the  modem  military  environment  over  the  past  decade  must  be 
accompanied  with  drastic  action  to  fix  the  problems  inherent  in  the  system. 

Implementation 

There  are  numerous  ways  in  which  to  combine  the  DT&E  and  OT&E  squadrons 
into  one  efficient  organization;  the  solution  discussed  here  is  only  one  of  them.  An  idea 
for  the  stmcture  and  makeup  of  the  proposed  organization  will  be  developed  and 
expanded  in  order  to  validate  the  concept. 

Stmcture 

Due  to  the  size  and  organizational  stmcture  currently  in  place,  the  basis  for  the 
combined  squadron  will  be  the  current  VX-31  organization.  Additionally,  the  DT&E 
process  is  inherently  more  flexible  as  requirements  are  generated  internal  to  the  program 
and  can  quickly  adapt  to  changes  in  a  spiral  development  process.  By  integrating  OT&E 
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into  this  same  structure,  OT&E  can  become  more  flexible  as  well.  For  the  purposes  of 
this  study  all  VX-9  assets  will  be  considered  for  integration  into  VX-3 1 . 

The  first  major  hurdle  with  integration  is  the  role  of  the  operational  evaluators  as 
an  independent  and  objective  “agency.”  Since  the  changes  proposed  do  not  affect 
COMOPTEVFOR  or  the  office  of  DOT&E,  the  independent  “agency”  still  exists.  What 
will  have  to  be  included  in  any  combined  squadron  is  a  way  to  gather  and  report 
evaluation  results  and  still  maintain  an  independent  status.  Additionally,  by  going  this 
route  the  requirement  to  have  Congressional  law  changed  has  been  removed  as  long  as  it 
can  be  shown  that  any  results  from  evaluations  are  independent  and  objective.  However, 
this  is  a  source  of  debate  but  most  state  that  DoD  has  the  authority  under  Title  10  USC. 

The  focus  of  discussion  will  be  on  adding  elements  to  the  current  structure  at  VX- 
31.  These  elements  are  direct  transfers  associated  with  the  closing  of  VX-9  and  the 
consolidation  of  assets.  Not  all  assets  will  have  to  be  transferred  as  will  be  shown  in  the 
course  of  discussion.  The  true  cost  savings  associated  with  this  reorganization  is  the 
elimination  of  overhead  associated  with  running  a  separate  squadron  and  the  reduction  in 
the  number  of  aircraft,  aircrew,  and  maintainers  required  to  conduct  the  operational 
evaluation  mission. 

VX-3 1  already  contains  a  robust  administrative  structure  on  conducting 
developmental  test  and  evaluation.  In  order  to  assume  the  new  operational  test  role, 
minor  additions  will  have  to  be  made  to  the  administrative  structure.  In  addition  to  the 
already  established  Commanding  Officer  (CO),  Chief  Test  Pilot  (CTP),  and  Executive 
Officer  (XO),  a  position  of  Operational  Test  Director  (OTD)  will  have  to  be  added.  This 
billet  will  be  a  commander/lieutenant  colonel  position.  The  CO  will  have  overall 
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responsibility  for  the  integrated  squadron.  The  CTP  will  assume  responsibility  for  the 
developmental  testing  while  the  OTD  will  be  responsible  for  the  operational  testing.  The 
role  of  the  OTD  will  be  to  run  all  operational  evaluations  to  be  eondueted  and  be  the 
administrative  eonduit  to  COTF. 

Airerew  will  be  integrated  into  the  new  strueture  as  well,  although  at  a  redueed 
number.  The  redueed  number  will  be  due  to  the  reduetion  of  aireraft  to  be  assigned  and  a 
eomplete  ehange  in  how  operational  test  is  to  be  eondueted.  Table  1  shows  the  eurrent 
staffing  levels  of  both  squadrons  with  a  representative  Fleet  F/A-18F  squadron  ineluded. 


Table  1.  Squadron  Staffing  Comparisons 


Squadron 

Pilots 

Weapons  System  Offieers 
(WSO’s) 

Other  Offieers 

Enlisted 

VX-9^^ 

23 

8 

8 

-250 

VX-31^' 

15 

7 

3 

322 

Fleet  F/A-18F^^ 

17 

18 

5 

194 

Source:  Created  by  author.  Information  extracted  from  squadron  documents. 

The  integrated  squadron  would  have  a  new  eore  of  10-15  operational  evaluators,  down 
from  over  30  airerew  previously  in  VX-9.  They  will  integrate  into  the  eurrent  projeet 
strueture  and  work  direetly  with  the  developmental  test  pilots  on  day  to  day  projeet 
funetions.  The  day  to  day  involvement  should  help  faeilitate  early  involvement  and  the 
insight  gained  from  being  involved  from  the  beginning  of  a  program  should  reduee  the 
seope  of  the  follow-on  dedieated  OT&E  and  eontribute  to  redueed  life  eyele  time  and 
eost.  Training  of  the  new  airerew  will  also  be  a  faetor  whieh  will  be  diseussed  later. 
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The  maintenance  departments  will  undergo  a  similar,  but  much  more  dramatic 
restructuring.  Table  1  also  shows  the  representative  maintenance  staffing  of  the 
squadrons  involved  and  a  representative  Fleet  squadron.  A  core  problem  when  integrating 
the  two  maintenance  departments  is  that  currently  VX-9  has  military  maintainers,  while 
VX-3 1  has  recently  moved  to  a  contract  maintenance  force  with  military  oversight.  As 
part  of  the  operational  evaluation  on  new  systems,  maintenance  practices  must  also  be 
evaluated  under  conditions  similar  to  Fleet  maintenance.  Because  of  this  requirement  a 
purely  contract  maintenance  force  is  not  a  viable  option.  However,  a  smaller  military 
maintenance  section  could  be  established  within  VX-3 1  ’s  current  structure  to  evaluate 
maintenance  practices  when  the  need  arose. 

To  accompany  the  integration  of  the  aircrew  and  maintenance,  aircraft  assets  will 
have  to  be  integrated.  Table  2  shows  the  current  aircraft  totals  for  all  Hornet  types 
currently  assigned  to  each  squadron. 


Table  2.  Current  Squadron  Aircraft  Allocation 


Squadron 

F/A-18C 

F/A-18D 

F/A-18E 

F/A-18F 

EA-18G 

VX-9^^ 

1 

2 

5 

7 

N/A 

VX-31^' 

3 

2 

3 

9 

2 

Fleet  F/A-18F 

- 

- 

- 

12 

- 

Source:  Created  by  author.  Information  extracted  from  squadron  documents. 


To  effectively  accomplish  most  tasks  in  an  integrated  squadron  will  take  the  addition  of 
approximately  six  aircraft.  These  aircraft  would  have  to  meet  the  test  requirements  VX-9 


aircraft  currently  meet  as  they  must  be  in  a  Fleet  representative  configuration.  These 
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aircraft  would  be  maintained  by  the  eombined  contractor/military  maintenance  of  the  new 
squadron.  When  operational  evaluations  oeeur,  a  team  of  military  maintainors  will 
execute  the  evaluation.  The  existing  maintenanee  personnel  would  have  to  be  inereased 
to  aeeommodate  the  addition  of  the  aircraft.  An  increase  of  ten  maintainors  of  various 
ratings  would  be  required  for  eaoh  airoraft  added  at  a  oost  of  $85,000  per  man  year.^^ 
Effeotively  this  combination  alone  would  reduce  the  ourrent  VX-9  foroe  of  airorew, 
maintainors,  and  airoraft  down  to  15,  60,  and  six  respeotively.  This  equates  to  a  reduotion 
of  52%,  76%,  and  60%  in  eaoh  oategory.  In  maintenanee  personnel  alone  the  savings 
would  be  over  16.2  million  dollars  per  year. 

The  ourrent  staff  of  VX-9  oontraotors  and  engineers  would  remain  unchanged. 
However,  their  role  in  the  new  struoture  would  differ  dramatioally.  The  personnel  would 
have  to  be  integrated  into  the  test  teams  already  established  on  the  developmental  side 
and  provide  input  to  the  developmental  team  on  operational  issues  and  oonoerns.  They 
would  also  have  to  step  up  their  test  preparation  work  in  order  to  meet  the  new 
requirements  assooiated  with  how  the  bulk  of  operational  evaluations  would  be 
eonducted  under  this  eonstruet. 

Training 

Training  for  airorew  and  engineers  is  extremely  important  in  understanding  and 
running  a  test  in  the  aoquisition  oommunity.  Well  trained  test  personnel  are  required  to 
aooomplish  test  programs  effioiently.  Today  this  training  is  rather  one-sided  as  only  the 
developmental  test  oommunity  has  a  well  defined  training  program.  All  aircrew  from 
both  DT&E  and  OT&E  have  a  similar  basis  of  experienoe  prior  to  their  assignments  in 

these  two  oommunities,  having  served  three  to  four  years  in  an  aotive  Eleet  squadron. 
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After  this  initial  shared  experienee  the  paths  for  training  diverge.  The  DT&E  airerew  and 
some  engineers  attend  an  1 1 -month  detailed  instruction  at  a  U.S.  or  foreign  test  pilot 
school  in  the  planning,  conduct,  and  execution  of  tests.  This  instruction  also  includes 
classes  on  the  makeup  of  the  acquisition  structure  along  with  Congressional  and  DoD 
requirements  of  the  acquisition  system.  By  contrast  no  such  equivalent  exists  for  OT&E 
aircrew.  Operational  evaluator  aircrew  may  attend  a  three  day  course  of  instruction  at 
COTE  but  this  requirement  is  often  bypassed.  Instruction  is  mainly  on  reporting 
requirements  and  structure  of  the  organization.  Once  both  developmental  and  operational 
aircrew  finish  their  respective  instruction  both  aircrew  have  the  requirements  of  the 
Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  to  maintain.  DAWIA 
requires  continued  study  in  the  field  of  acquisition  tailored  to  a  primary  subspecialty  area. 
This  training  is  offered  online  and  at  resident  classes  via  the  Defense  Acquisition 
University  (DAU).  The  requirements  for  all  aircrew  are  the  same  and  based  on  rank.  In 
an  integrated  squadron  a  better  training  regimen  must  be  established. 

This  study  does  not  suggest  that  everyone  should  be  trained  to  be  a  rated  test  pilot. 
However,  it  is  recommended  that  a  new  course  of  instruction  be  offered  at  the  US  Navy 
Test  Pilot  School  (USNTPS)  similar  to  the  one  offered  at  the  US  Air  Eorce  Test  Pilot 
School  (USAETPS).  USAETPS  offers  a  two  week  course  of  instruction  at  their  school 
which  is  given  specifically  for  operational  evaluators.  USNTPS  already  offers  “short 
courses”  to  members  of  the  acquisition  community  which  last  from  one  to  a  few  weeks 
on  varying  topics  of  instruction.  USNTPS  should  take  the  program  currently  offered  at 
COTE  and  integrate  it  into  a  “short  course.”  This  operational  evaluation  “short  course” 
could  also  have  elements  implemented  into  the  core  curriculum  of  the  test  pilot 
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instruction  program  to  enhance  all  aircrew  learning.  The  new  “short  course”  would  be  a 
requirement  for  any  airerew  that  have  not  completed  the  longer  test  pilot  course  of 
instruction.  An  added  benefit  of  reloeating  the  operational  evaluation  training  to  USNTPS 
would  be  the  eombining  of  DT&E  and  OT&E  training  at  a  single  location  . 

So  what  does  that  leave  in  numbers?  Eor  the  theoretical  staffing  of  15  operational 
evaluation  aircrew  the  recommendation  would  be  that  five  to  eight  of  those  be  test  pilot 
school  graduates  to  help  in  the  conduct  and  execution  of  tests.  Additionally  this  would 
open  up  more  flight  opportunities  to  these  airerew  as  they  would  have  the  required 
training  to  exeeute  some  of  the  more  demanding  tests.  An  additional  three  to  five  airerew 
would  be  El.S.  Navy  Eighter  Weapons  Sehool  (TOPGETN)  graduates.  The  remaining 
airerew  would  be  direct  from  operational  Eleet  tours.  Therefore  both  the  TOPGETN  and 
Eleet  airerew  would  be  eandidates  for  the  USNTPS  “short  course”  along  with  any 
eontraetors  or  engineers. 

Conduct  of  Test 

The  addition  of  aircraft  and  aircrew  would  greatly  enhanee  the  ability  of  the 

developmental  test  team  to  more  efficiently  use  instrumented  test  aircraft  and  allow  for 

early  involvement  of  operational  evaluators  in  the  early  stages  of  testing.  Early 

involvement  is  a  key  tenet  of  operational  evaluation.  The  longer  a  program  progresses 

down  the  aequisition  life  eycle,  the  eost  and  time  to  fix  any  problems  found  inerease 

exponentially.  Decisions  made  in  the  first  10  to  15  percent  of  the  work  done  on  a 

program  usually  drive  the  program  costs.  OT&E  has  a  good  track  record  over  the  past 

decade  of  getting  involved  early  in  major  test  programs  of  new  equipment.  A  primary 

example  of  this  suecess  is  the  EA-18G  Growler  program,  where  DT&E  and  OT&E 
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aircrew  work  side  by  side  on  a  near  daily  basis  to  eonduet  both  developmental  and 
operational  testing  and  evaluations.  Yet  after  the  new  equipment  has  been  fielded  and  the 
novelty  of  working  on  a  new  program  wears  off,  OT&E  early  involvement  in  programs 
beeomes  nearly  non-existent.  However,  due  to  the  DoD’s  preferred  aequisition  proeess  of 
spiral  development,  this  leaves  major  upgrades  to  systems  with  no  OT&E  early 
involvement.  Eor  many  programs  the  first  OT&E  interaetion  is  at  the  Operational  Test 
Readiness  Review  (OTRR)  whieh  is  far  too  late  in  any  program  to  make  major  deeisions 
should  problems  oecur.  Charles  MeQueary,  DOT&E,  states  “Developmental  testing  is  the 
plaee  to  find  problems.  Operational  testing  should  be  a  period  of  eonfirmation,  not  a 
period  of  diseovery.”^^  In  order  to  do  this  OT&E  must  be  involved  early  and 
eontinuously. 

The  integrated  eoneept  will  require  some  major  ehanges  in  how  operational 
evaluations  would  be  condueted.  Traditionally  after  OTRR,  the  testing  phase  would  be 
transferred  to  VX-9  and  they  would  eonduet  the  operational  evaluation.  Since  the  size  of 
the  operational  foree  of  the  integrated  squadron  has  been  greatly  redueed  this  is  no  longer 
a  viable  option.  A  new  way  to  eonduet  the  operational  evaluation  must  be  used.  A  major 
finding  by  the  2005  COMOPTEVEOR  report  was  to  “explore  the  opportunities  presented 
by  the  eonduet  of  testing  during  fleet  training  and  dedieated  fleet  experimentation 
events. 

Traditionally,  Eleet  squadrons  returning  from  deployment  enter  a  period  of 
redueed  operational  tempo  after  their  return,  prior  to  beginning  training  for  the  next 
deployment  eyele.  During  this  down  time  flights  are  still  eondueted  at  a  redueed  pace. 
Typieally  towards  the  end  of  the  redueed  sehedule  time,  the  squadron  sehedules  a 
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detachment  to  a  location  away  from  home  station  to  get  back  into  the  training  cycle. 

These  detachments  usually  are  to  places  like  Key  West,  Florida  and  Hawaii.  Normally 
these  areas  have  sufficient  training  ranges  and  adversary  squadron  support.  If  adversaries 
are  not  available  on  site  then  arrangements  are  normally  made  to  coordinate  a  detachment 
with  another  squadron  to  swap  adversary  support.  These  returning  squadrons  offer  a 
unique  opportunity  to  conduct  the  operational  evaluation.  Who  is  better  qualified  in 
current  Fleet  operations  and  tactics  than  those  squadrons  returning  from  deployment? 

This  may  sound  like  a  novel  idea  but  it  is  not  new.  VX-9  has  used  a  similar  policy  to 
conduct  operational  evaluations  before  when  they  were  unable  to  complete  the  mission 
themselves.  The  most  recent  example  was  the  Software  Configuration  Set  (SCS)  20X 
testing  done  by  VFA-15  in  Oceana,  Virginia.  In  this  case  a  “trusted  agent”  agreement  was 
given  to  the  Fleet  squadron  for  testing  and  VX-9  and  TOPGUN  sent  liaisons  to  VFA-15 
to  help  in  the  conduct  of  the  evaluation. 

While  this  example  offers  a  glimpse  at  how  operational  evaluations  would  be 
done  in  the  integrated  squadron,  some  things  must  change  in  how  the  SCS  20X 
evaluation  was  conducted.  Historically  the  “trusted  agent”  status  for  testing  only  allows 
the  given  squadron  to  fly  with  the  new  equipment  during  the  test  phase.  This  is  also  a 
problem  associated  with  testing  at  VX-9  as  well.  The  longer  and  more  often  a  particular 
test  item  is  flown,  the  greater  the  chance  of  uncovering,  and  then  fixing,  a  problem.  The 
example  of  a  returning  squadron  from  deployment  will  form  the  basis  of  the  analysis. 

The  returning  squadron  could  be  given  a  new  software  load  prior  to  their  return  to  home 
station.  During  the  transit  back  from  theater  is  an  ideal  time  to  conduct  orientation 
training  on  the  items  to  be  evaluated  since  all  squadron  members  are  present  and  there  is 
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ample  time.  Airerew  from  the  test  squadron  eould  be  sent  aboard  to  eonduet  the  training 
for  both  the  pilots  and  maintenance  personnel.  All  available  documentation  and  points  of 
contact  would  be  delivered  so  the  squadron  could  become  familiar  with  the  items  under 
evaluation.  Upon  return  to  home  station  the  item  under  evaluation  (new  software,  a  new 
avionics  box,  or  other  item)  would  arrive.  The  aircrew  would  then  begin  flying  with  the 
items  as  soon  as  practical  so  they  can  increase  their  familiarization  with  the  system. 

During  the  familiarization  period  aircrew  support  would  be  provided  from  the  test 
squadron  to  enhance  the  learning  process.  This  would  typically  give  a  one  to  two  month 
window  prior  to  the  full  evaluation  for  aircrew  to  get  familiar  with  the  system  and  its 
differences.  This  in-depth  knowledge  of  system  function  will  pay  enormous  dividends  in 
testing.  Once  the  official  evaluation  period  begins,  the  Fleet  squadron  would  be  joined  by 
members  from  the  test  squadron  and  TOPGUN  to  help  conduct  the  evaluation.  The 
evaluation  would  be  run  by  the  operational  flight  test  engineers  for  data  gathering  and 
reporting.  After  the  evaluation  is  complete  the  squadron  should  be  allowed  to  continue 
flying  the  items  (where  practical)  instead  of  making  them  revert  back  to  older  variants 
until  the  item  is  subsequently  released  to  the  entire  Fleet.  In  all,  this  would  give 
approximately  six  months  of  extra  flight  time  on  the  systems  under  evaluation. 

Since  current  OT&E  aircrew  require  no  specialized  training  in  flight  test,  the  Fleet 
aircrew  would  likewise  not  require  any  training  other  than  system  familiarization.  During 
the  operational  evaluation,  aircrew  and  engineers  from  the  integrated  test  squadron  would 
be  planning  and  executing  the  test,  flying  with  the  Fleet  aircrew,  gathering  test  data,  and 
submitting  the  evaluation  report  to  COTF.  Some  test  points  may  require  more 
experienced  aircrew,  but  nothing  the  Fleet  squadron  could  not  provide. 
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The  cost  associated  with  conducting  the  operational  evaluation  would  be  taken 
from  program  funding  as  it  would  normally  have  if  the  evaluation  were  done  under  the 
current  system.  The  advantage  of  combining  this  effort  with  a  Fleet  squadron  is  that  the 
Fleet  squadron  would  have  completed  a  detachment  anyway  and  funding  would  have 
been  taken  out  of  their  squadron  operational  funds  for  training.  The  detachment  still 
accomplishes  the  training  requirements  of  the  squadron  and  saves  the  Navy  money  by  not 
having  to  fund  additional  detachments. 

The  coordination  of  schedules  would  require  some  effort.  Every  year  three  to  four 
Carrier  Air  Wings  complete  the  training  and  deployment  cycle.  Each  Air  Wing  would 
have  four  Hornet  squadrons  of  varying  makeup  to  accomplish  testing.  Therefore,  roughly 
every  three  months  there  would  be  an  opportunity  to  participate  in  a  flight  test.  This 
schedule  may  slide  by  a  few  months  either  direction  but  the  opportunities  to  match  Elect 
schedules  to  program  schedules  are  available. 

While  this  takes  care  of  the  majority  of  the  evaluation,  the  integrated  squadrons’ 
operational  side  must  conduct  some  testing  prior  to  readiness  for  the  Elect  evaluation. 

The  only  change  to  current  procedure  required  here  would  be  when  the  operationally 
configured  aircraft  fly  with  the  new  systems.  Historically  a  system  must  be  in  its  “final” 
configuration  before  it  is  loaded  into  the  operationally  configured  aircraft.  While  this  is 
good  in  principle,  it  may  not  be  relative  in  real  life.  Due  to  spiral  development  the 
configuration  is  never  set  as  final  until  released  to  the  Elect.  Again  this  mindset  limits 
exposure  time  for  uncovering  possible  problems. 


54 


Supporting  Agencies 

This  improved  conduct  of  test  will  greatly  enhance  efficiency  and  communication 
between  the  two  entities  and  produce  a  better  product  overall  for  the  Fleet.  However  it 
does  not  address  some  other  issues  that  surface  due  to  the  elimination  of  VX-9. 
Operational  evaluation  is  not  the  only  mission  VX-9  conducts.  The  mission  of  developing 
new  tactics  is  one  which  is  very  important  to  the  use  of  the  new  systems.  Recent  history 
has  shown  there  has  been  a  lag  in  the  deployment  of  new  systems  and  the  tactics  of  using 
them.  While  it  may  not  seem  so  at  first,  the  elimination  of  VX-9  might  help  resolve  this 
problem. 

TOPGUN  is  the  purveyor  of  tactics  to  the  naval  aviation  Fleet.  The  work  they 
conduct  and  the  products  they  produce  form  the  back  bone  of  the  way  naval  aviation 
operates.  However,  their  limited  resources  have  meant  they  could  not  develop  tactics 
until  after  Fleet  introduction  and  Fleet  systems  started  rotating  through  the  TOPGUN 
classes.  This  leaves  gaps  in  training  and  employment  of  the  new  systems.  Primary 
examples  of  this  in  recent  history  have  been  the  development  of  roles  and  responsibilities 
for  aircrew  piloting  the  F/A-18F  two-seat  aircraft,  and  the  tactics  for  operating  the  Active 
Electronically  Scanned  Array  (AESA)  radar.  While  the  Navy  aggressively  deploys  the 
new  AESA  equipped  aircraft,  tactics  still  have  not  been  fully  developed. 

The  elimination  of  VX-9  frees  up  both  aircraft  and  aircrew  for  assignment 
elsewhere.  In  order  to  accomplish  the  mission  of  tactics  development  the  best  fit  for  these 
aircraft  would  be  as  assigned  to  TOPGUN.  These  aircraft  could  be  loaded  with  the  same 
systems  currently  under  test  that  have  matured  to  the  point  where  they  would  have 
previously  been  released  for  OT&E  flight.  This  would  address  any  safety  and 
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airworthiness  issues  and  enable  their  use  for  development  of  operational  taeties.  Another 
benefit  to  this  would  be  additional  sorties  and  flight  time  on  the  software  in  an 
operational  environment  to  diseover  any  errors  or  improvements.  The  addition  of  more 
TOPGUN  trained  personnel  at  the  integrated  squadron  would  help  liaise  with  members  at 
TOPGUN.  The  eommunieation  that  exists  eurrently  between  former  and  eurrent 
TOPGUN  members  is  extraordinary.  Now  TOPGUN  trained  personnel  will  be  integrated 
into  the  test  proeesses  early,  whieh  will  pay  huge  dividends  for  new  and  existing 
programs  under  development.  In  order  to  get  the  most  out  of  this  relationship,  TOPGUN 
should  institute  some  training  in  their  eore  eurrieulum  on  the  Test  and  Evaluation  (T&E) 
proeess. 

Advantages 

There  are  numerous  advantages  of  an  integrated  squadron  eoneept.  There  are  also 
some  drawbaeks  assoeiated  with  the  eoneept,  but  those  eould  be  minimized  through 
support  from  other  eommands.  To  evaluate  the  utility  of  the  eoneept,  the  problems  with 
the  existing  strueture  would  be  analyzed. 

Cost  and  Sehedule 

Traditionally  the  eost  and  sehedule  of  the  operational  evaluation  of  any  program 
is  minimal  to  the  overall  eost  and  sehedule  of  the  development.  Table  3  shows  a  list  of 
reeently  eompleted  Air  Eoree  aequisition  programs  and  the  eost  of  development. 
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Table  3.  Air  Force  OT&E  Cost  Data 


Program 

Acquisition  Cost 

OT&E  Cost 

%  OT&E 

Joint  Direct  Attack  Munition 
(JDAM)  (Mk-84) 

2,386  M 

2.0  M 

0.085% 

Joint  Helmet  Mounted  Cueing  System 
(JHMCS) 

569  M 

l.OM 

0.179% 

B-1  Conventional  Mission  Upgrade 
Program  (CMUP)  D 

669M 

0.9  M 

0.148% 

B-1  Conventional  Mission  Upgrade 
Program  (CMUP)  E 

840  M 

0.3  M 

0.041% 

Cheyenne  Mountain  Upgrade 

1,800  M 

2.1  M 

0.118% 

APG-63  Radar  Upgrade 

1,000  M 

1.6M 

0.160% 

Wind  Corrected  Munitions  Dispenser 
(WCMD) 

650  M 

1.2  M 

0.186% 

Joint  Stand-Off  Weapon  (JSOW) 

6,000  M 

2.2  M 

0.036% 

Total 

7,915  M 

9.34  M 

0.118% 

Source:  Christie,  Thomas  P.,  “T&E  Transformation”  International  Test  &  Evaluation  Summit  & 
Exhibition,  March  2003:  9. 


As  shown  in  the  table,  the  eost  of  OT&E  was  less  than  0.2%  of  the  total  program  eost. 
Therefore,  just  by  integrating  the  squadrons  a  huge  savings  will  not  be  apparent  in  total 
program  eost.  The  near  term  objective  should  not  be  just  about  reducing  cost,  but  to 

-5  •3 

reduce  waste,  therefore  allowing  for  more  productive  use  of  the  same  funding.  In  the 
long  term,  this  will  drive  down  cost.  Similarly,  the  time  savings  of  just  integrating  will 
not  be  very  apparent.  Table  4  shows  a  comparison  of  recent  aircraft  and  weapons 
acquisition  programs. 
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Table  4.  Major  System  Test  Time  Comparison 


System 

Development  Time 

Planned  Eength  of  lOT&E 

E-22 

22  years 

6  months 

V-22 

20  years 

7  months 

Joint  Strike  Fighter  (JSF) 

15  years  (Planned) 

- 

AIM-9X 

5  years 

6  months 

Source:  Christie,  Thomas  P.,  “T&E  Transformation”  International  Test  &  Evaluation  Summit  & 
Exhibition,  March  2003 :  1 1 . 


As  shown,  OT&E  is  a  relatively  short  period  in  the  overall  development  of  the  program. 
Typieal  delays  in  OT  are  on  the  order  of  months,  which  is  small  in  comparison  to  the 
longer  running  development  of  programs.  In  both  cases  it  is  shown  that  you  do  not  save 
time  or  money  by  cutting  tests.  However,  in  a  recent  study  conducted  by  COTF, 
preliminary  calculations  showed  that  by  instituting  just  some  of  the  solutions  mentioned 
here,  a  15  percent  reduction  in  T&E  time  could  be  achieved.  In  a  program  length  of  20 
years  this  equates  to  3  years. 

The  key  cost  saver  in  integrating  the  squadrons  would  be  the  elimination  of  the 
overhead  associated  with  another  squadron.  While  some  of  the  infrastructure  would  be 
added  elsewhere  for  support,  the  increase  in  efficiency  and  elimination  of  duplication  of 
tests  will  shorten  program  deployment  times  by  months  rather  than  years. 

The  cost  savings  assumes  it  would  not  be  more  expensive  to  farm-out  operational 
evaluation  to  returning  Elect  units.  Table  5  shows  a  cost  breakdown  of  the  cost  per  flight 
hour  to  operate  an  E/A-18E/E  Super  Hornet. 
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Table  5.  F/A-18F  Cost  Comparison  Data 

($  per  Flight  Hour) 


Squadron 

Fixed  Cost 

AVDFR 

Maintenance/Parts 

VX-9^^ 

3161.95 

3582.13 

1544.97 

VX-31^'’ 

3236 

2570 

4544 

Fleet  F/A-18F^^ 

3115.76 

2682.53 

1070.26 

Note  1-  Aviation  Depot  Level  Repairable 
Source:  Created  by  author.  Information  extracted  from  flight  hour  analysis  reports. 


The  fixed  cost  to  operate  in  any  squadron  is  roughly  the  same  across  the  board.  The  table 
highlights  differences  in  two  areas:  AVDLR  and  Maintenance/Parts.  VX-31  is  unique  in 
that  its  AVDLR  costs  are  so  low  due  to  the  way  in  which  aircraft  are  maintained.  Each 
aircraft  is  a  specialized  aircraft  and  not  of  a  Fleet  representative  configuration.  Therefore, 
most  maintenance  is  done  on  site  and  not  sent  to  a  depot.  Additionally,  maintenance/parts 
are  so  high  for  VX-31  is  for  two  reasons:  specialized  one-of-a-kind  parts  and  contract 
maintenance.  What  is  not  reflected  in  this  high  number  is  the  personnel  cost  savings 
generated  by  shifting  from  a  military  maintenance  force  to  contract  personnel  which 
skews  this  number  heavily.  Integrating  to  a  VX-3 1  structure  for  the  integrated  squadron 
would  not  change  the  numbers  significantly.  Cost  savings  would  be  seen  in  the  reduction 
in  number  of  aircraft  and  using  the  lower  cost  Fleet  aircraft  to  perform  the  majority  of 
testing.  Additionally,  the  greatest  cost  savings  will  be  in  eliminating  institutional  and 
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overhead  costs  as  those  are  funded  outside  of  acquisition  program  budgets. 
Communication 

Probably  the  largest  source  of  problems  within  the  current  structure  is  the 
communication  problem.  The  two  separate  chains  of  command  are  barriers  that  stove- 
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pipe  information  and  become  obstacles  to  rational  and  efficient  decision  making. 
Cooperation  is  more  effective  than  confrontation.  The  singular  structure  fixes  this 
enormous  problem.  By  integrating  everyone  under  one  chain  of  command  and  working 
side  by  side  on  a  daily  basis,  the  barriers  are  lifted.  While  there  may  still  be  internal  strife 
between  the  developmental  and  operational  sides  of  the  integrated  squadron,  there  is  a 
means  by  which  to  solve  these  issues  through  the  CTP,  OTD,  and  their  higher  entities 
within  the  acquisition  process.  In  the  increased  complexity  of  future  programs,  OT&E’s 
early  and  continuous  involvement  will  be  critical,  and  effective  communication  will  be 
essential  to  program  efficiency  and  success.  This  solution  achieves  that  goal. 

Personnel 

Both  squadrons  currently  are  staffed  well  below  the  required  levels.  The  idea  of 
“do  more  with  less”  has  become  a  mainstay  in  the  military  community.  While  these 
staffing  problems  have  been  evident  for  some  time  the  programs  still  get  completed.  The 
integration  of  the  two  squadrons  will  by  no  means  fix  the  staffing  issues  currently 
plaguing  the  Navy  as  a  whole.  It  will,  however  help  towards  achieving  a  more 
manageable  solution.  Sharing  manpower  between  contractor,  developmental,  and 
operational  testers  would  help  address  the  resource  shortfalls."^*’  Simply  combining  the 
two  squadrons  together  with  the  same  amount  of  total  aircrew  is  not  the  answer  either. 
This  would  suffer  severely  in  efficiency  of  test  conduct  and  operation.  An  in-depth 
analysis  of  the  optimal  squadron  staffing  is  required. 
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Test  Structure 


The  proposed  solution  for  conducting  the  operational  evaluation  is  not  a  new  one. 
Having  Fleet  squadrons  conduct  the  evaluation  has  been  done  for  some  time.  However, 
improvements  to  the  practice  could  yield  huge  dividends  to  both  the  test  community  and 
the  Fleet.  Allowing  the  Fleet  evaluation  squadrons,  TOPGUN,  and  the  operational  testers 
to  get  the  items  under  test  sooner  than  would  normally  be  achieved,  would  benefit  the 
entire  system.  The  test  community  benefits  from  the  increased  sorties,  hours,  and  “eyes 
on”  any  given  system  under  test.  The  Fleet  benefits  through  the  ability  of  TOPGUN  to 
have  tactics  ready  upon  Fleet  release  and  deployment,  and  through  the  satisfaction  of 
having  a  voice  in  the  process  of  development.  The  structure  of  the  integrated  squadron 
also  helps  in  cutting  down  the  redundancy  of  test  as  all  members,  both  developmental  and 
operational,  have  input  from  the  start  of  a  program. 

Summary 

Chapter  Four  outlined  a  possible  solution  to  the  problems  noted  in  Chapter  Two. 
This  solution  is  but  one  of  many  possible  solutions  available  to  combine  the  DT&E  and 
OT&E  test  squadrons  at  China  Eake,  California.  This  study  has  examined  the  historical 
relationships  between  the  two  squadrons,  the  legacy  of  acquisition  reform,  the  current  test 
structure  and  its  problems,  and  a  possible  solution  on  how  to  fix  them.  However,  a  more 
in-depth  study  is  needed. 
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CHAPTER  5 


CONCLUSIONS  AND  RECOMMENDATIONS 

Review 

The  preeeding  ehapters  have  outlined  the  history,  problems,  reform  efforts,  and  a 
proposed  solution  for  solving  the  problems  assoeiated  with  the  relationship  between  the 
naval  strike  fighter  DT&E  and  OT&E  eommunities.  The  following  is  a  brief  synopsis  of 
the  researeh. 

Chapter  One  began  the  discussion  with  a  brief  history  of  the  acquisition  reform 
efforts,  particularly  efforts  within  the  past  50  years.  The  problems  associated  with 
legislative  and  budgetary  control  by  Congress  and  the  pure  size  of  the  acquisition  system 
were  also  detailed.  The  issues  of  acquisition  program  cost,  schedule,  communication,  and 
personnel  were  introduced.  The  discussion  culminated  with  the  primary  thesis  of  the 
paper:  to  determine  how  the  relationship  between  the  developmental  and  operational 
aircraft  testing  can  be  changed  to  better  align  to  the  realities  of  cost  and  schedule  in  the 
current  testing  environment. 

The  literature  review  in  Chapter  Two  examined  the  body  of  works  available  on 
the  subject.  A  brief  history  of  the  DT&E  and  OT&E  squadrons  involved,  as  well  as  the 
current  test  structure  was  discussed.  The  subject  of  transformation  was  introduced  and  the 
reasons  why  reform  was  necessary  and  the  resistance  to  that  reform  were  debated.  The 
problem  areas  noted  in  Chapter  One,  cost,  schedule,  communication,  and  personnel,  were 
expanded  on  and  the  problems  with  these  areas  within  the  current  system  were  analyzed. 
Einally,  a  few  attempts  by  the  Navy  and  Air  Eorce  at  fixing  the  issues  identified 
previously  were  analyzed. 
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Chapter  Three  diseussed  the  methodology  used  to  analyze  the  primary  researeh 
question.  Researeh  consisted  of  an  historical  examination,  current  acquisition  process 
analysis,  and  then  an  analysis  of  the  problems  identified  through  that  research.  To  arrive 
at  a  final  solution  the  problem  areas  identified  in  Chapter  One  were  analyzed. 

Finally,  Chapter  Four  provided  analysis  and  comparison  of  the  three  options 
available  to  reform  the  current  system;  do  nothing,  eliminate  the  OT&E  squadron,  or 
combine  the  DT&E  and  OT&E  squadrons  into  one  entity.  The  problem  areas  identified  in 
Chapter  1  were  used  to  compare  the  alternatives.  The  solution  of  combining  DT&E  and 
OT&E  into  a  single  entity  was  then  expanded  to  answer  the  primary  research  question  of 
determining  how  the  relationship  between  DT&E  and  OT&E  could  be  changed  to  better 
align  to  the  realities  of  the  current  testing  environment. 

Conclusion 

“Our  Bottom  line  is. .  .field  a  superior  capability,  affordably  and  in  less  time  than 
our  potential  adversaries.”  -Honorable  Paul  G.  Kaminski,  Under  Secretary  of 
Defense  for  Acquisition  and  Technology.' 

This  paper  has  presented  a  solution  of  combining  the  DT&E  and  OT&E 
squadrons  located  at  China  Eake,  California  to  solve  the  problems  of  cost,  schedule, 
communication,  personnel,  and  test  conduct  related  to  the  current  structure.  Through 
eliminating  the  current  OT&E  squadron,  and  placing  some  of  their  assets  into  the  DT&E 
squadron  to  assume  the  operational  evaluation  mission,  it  was  shown  that  a  substantial 
increase  in  efficiency  could  be  gained.  Determining  the  exact  value  in  time  and  money  is 
well  beyond  the  scope  of  this  investigation  and  would  require  additional  focused 
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research.  The  concept  of  combining  DT&E  and  OT&E  into  a  singular  entity  was  shown 
to  be  valid  and  elaborated  more  on  ideas  presented  in  previous  studies. 

The  study  concluded  that  a  combined  squadron  would  require  some 
reorganization  due  to  the  assumption  of  the  operational  evaluation  role.  An  addition  of  15 
aircrew,  60  maintainers,  and  six  aircraft  to  the  current  DT&E  squadron  were  presented  as 
a  starting  point  for  reorganization.  These  additions,  with  the  elimination  of  the  OT&E 
squadron,  represent  a  reduction  of  52%  for  aircrew,  76%  for  maintainers,  and  60%  for 
aircraft.  In  maintenance  personnel  alone  the  savings  would  be  over  16.2  million  dollars 
per  year.  Eurther  research  is  required  to  determine  the  optimal  numbers  to  add  to  DT&E 
to  assume  the  operational  evaluation  role.  Once  that  is  determined,  additional  research 
could  be  done  to  conclude  the  savings  that  would  be  generated. 

•5 

Well  trained  test  personnel  are  required  to  accomplish  test  programs  efficiently. 
This  study  concluded  that  combining  DT&E  with  OT&E  would  require  minor  changes  in 
how  aircrew  and  engineers  are  trained.  The  solution  presented  was  to  model  the  Air 
Eorce  OT&E  training  program  at  EISAETPS  and  create  a  “short  course”  of  instruction  at 
USNTPS.  The  “short  course”  would  be  targeted  at  those  aircrew  and  engineers  that  do 
not  attend  the  longer  test  pilot  course  of  instruction.  This  collocation  of  training  would 
have  the  added  benefit  of  combining  DT&E  and  OT&E  training  at  a  single  source. 
Additionally,  the  paper  concluded  that  T&E  training  be  included  in  the  TOPGETN 
curriculum. 

The  change  to  the  way  in  which  OT&E  testing  is  conducted  is  the  most  far 
reaching  aspect  of  this  paper.  This  paper  concluded  that  by  using  Elect  assets  to  conduct  a 
vast  majority  of  the  OT&E  portion  of  test,  it  would  allow  for  the  drastic  reduction  in 


67 


personnel  and  overhead  assoeiated  with  the  combination  of  the  DT&E  and  OT&E 
squadrons.  Through  providing  support,  expertise,  and  planning  to  Elect  assets,  the 
operational  evaluation  could  be  conducted  in  less  time  and  would  provide  valuable  input 
to  product  development  and  expand  Fleet  system  knowledge.  This  effort  would  require 
the  support  of  Fleet  units  and  TOPGETN  in  order  to  work  effectively.  Further  study  is 
required  to  determine  the  actual  test  conduct  and  timeline  using  this  model. 

The  solutions  presented  go  a  long  way  in  solving  the  problems  of  cost,  schedule, 
communication,  personnel,  and  test  conduct.  This  study  only  scratches  the  surface  of 
what  could  be  achieved.  With  further  study  a  viable  working  solution  could  be 
developed. 

Today’s  acquisition  reforms  continue  the  trends  of  the  last  230  years.  What  is 
needed  is  a  drastic  change  to  the  existing  structure  to  spur  development  and  innovation. 
The  ideas  expressed  in  this  paper  only  touch  the  surface  of  what  could  be  accomplished 
through  integration  of  the  developmental  and  operational  test  squadrons.  The  solution 
presented  is  a  step  towards  achieving  the  Chief  of  Naval  Operations’  (CNO)  goal  of 
reducing  T&E  cost  by  20%.^  Experience  in  acquisition  reform  has  shown  there  will  be  no 
quick  fixes  and  there  will  be  many  obstacles  to  accomplishing  institutional  change.  The 
greatest  problem  for  implementing  this  idea  is  to  overcome  the  paradigms  present  in  the 
two  test  communities.  Institutions  have  a  vested  interest  in  maintaining  the  system  as  it  is, 
and  true  change  must  be  mandated  by  senior  management.^  Change  must  occur  via  a 
strong  proponent  and  leader  from  the  top,  and  buy-in  must  be  attained  from  all  levels  of 
leadership  on  down.  No  matter  which  T&E  organization  is  involved,  partnerships  with 
others  is  the  key  to  that  organization’s  future  relevance.^  If  these  changes  occur  the  Navy 
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will  have  an  efficient,  well  functioning  test  organization  that  produces  a  far  superior 
product  to  Fleet  aviators  in  less  time  and  at  a  greatly  reduced  cost.  However,  before  any 
meaningful  work  towards  this  implementation  can  be  done,  further  in-depth  study  is 
required  to  determine  the  actual  cost  effectiveness  of  this  approach.  The  following 
recommendations  are  made  for  the  implementation  of  this  solution. 

Recommendations 

•  The  numbers  presented  in  this  document  are  purely  an  observational  solution.  An 
in-depth  study  should  be  undertaken  in  the  following  areas: 

o  The  cost  savings  that  could  be  achieved  by  combining  the  two  squadrons, 
o  A  comprehensive  manpower  survey  on  what  the  optimal  staffing  of  the 
proposed  squadron  would  entail. 

o  A  usage  analysis  to  determine  the  optimal  aircraft  numbers  and  types  to  be 
added  to  VX-3 1  to  operate  in  the  operational  evaluation  role, 
o  USNTPS  in  conjunction  with  COTF  and  the  USAFTPS  develop  a  “short 
course”  for  training  operational  evaluators  and  integrate  portions  of  that 
course  into  the  core  USNTPS  curriculum, 
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